-- Logs begin at Mon 2022-07-25 11:56:37 UTC, end at Mon 2022-07-25 13:10:46 UTC. -- Jul 25 12:25:26.712413 dv-cinder-825492-28-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 25 12:25:26.720885 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 25 12:25:26.722932 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 25 12:25:26 2022] *** Jul 25 12:25:26.722932 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 25 12:25:26.722932 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 25 12:25:26.722932 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: nodename: dv-cinder-825492-28-iscsi Jul 25 12:25:26.722932 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: machine: x86_64 Jul 25 12:25:26.722932 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: clock source: unix Jul 25 12:25:26.722932 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: pcre jit disabled Jul 25 12:25:26.722932 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: detected number of CPU cores: 4 Jul 25 12:25:26.722932 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: current working directory: / Jul 25 12:25:26.722932 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: detected binary path: /usr/bin/uwsgi-core Jul 25 12:25:26.722932 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: your processes number limit is 47680 Jul 25 12:25:26.723536 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: your memory page size is 4096 bytes Jul 25 12:25:26.723536 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: detected max file descriptor number: 2048 Jul 25 12:25:26.723536 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: lock engine: pthread robust mutexes Jul 25 12:25:26.723536 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: thunder lock: enabled Jul 25 12:25:26.723536 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 25 12:25:26.723536 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 25 12:25:26.762480 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: Python main interpreter initialized at 0x55de56ee09c0 Jul 25 12:25:26.762480 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: python threads support enabled Jul 25 12:25:26.762480 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: your server socket listen backlog is limited to 100 connections Jul 25 12:25:26.762480 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: your mercy for graceful operations on workers is 90 seconds Jul 25 12:25:26.762803 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: mapped 403077 bytes (393 KB) for 2 cores Jul 25 12:25:26.762879 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: *** Operational MODE: preforking *** Jul 25 12:25:26.763149 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: *** uWSGI is running in multiple interpreter mode *** Jul 25 12:25:26.763149 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: spawned uWSGI master process (pid: 80316) Jul 25 12:25:26.763338 dv-cinder-825492-28-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 25 12:25:26.764370 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: spawned uWSGI worker 1 (pid: 80326, cores: 1) Jul 25 12:25:26.764370 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: spawned uWSGI worker 2 (pid: 80327, cores: 1) Jul 25 12:25:26.764370 dv-cinder-825492-28-iscsi devstack@keystone.service[80316]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 25 12:25:32.222783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 12:25:32.222783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: from cryptography.utils import int_from_bytes Jul 25 12:25:32.223871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 12:25:32.223871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: from cryptography.utils import int_from_bytes Jul 25 12:25:32.223996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 12:25:32.223996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: from cryptography.utils import int_from_bytes Jul 25 12:25:32.224493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 12:25:32.224493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: from cryptography.utils import int_from_bytes Jul 25 12:25:32.272249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 12:25:32.272439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 12:25:32.272600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 12:25:32.272701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] command line args: [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 12:25:32.272781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 12:25:32.272865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 12:25:32.272972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] command line args: [] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 12:25:32.273057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ================================================================================ {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 12:25:32.273242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 12:25:32.273319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] admin_token = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.273418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ================================================================================ {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 12:25:32.273508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] config_dir = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.273670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.273753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] admin_token = **** {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.273851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] config_source = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.273947 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] config_dir = [] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.274036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] debug = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.274114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.274239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.275567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] config_source = [] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.275567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] debug = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.275700 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.275760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.275842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.275907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.275965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_config_append = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] list_limit = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] log_config_append = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_dir = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_file = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.276752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] log_dir = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.277408 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.277482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] log_file = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.277482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.277591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.277653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.277722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.277785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.277848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.277848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.277994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.278064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.278130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.278130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.278249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.278414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.278490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.278607 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.278674 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.278835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.278969 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.279051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.279166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.279280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.279363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.279509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.279586 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.279751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.279829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.279996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.280115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.280267 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.280495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.280676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] publish_errors = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.280802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.280924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.281037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] publish_errors = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.281160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.281274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.281388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.281499 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.281499 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.281715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.281850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.282001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.282123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.282244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] standard_threads = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.282425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] standard_threads = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.282550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.282671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.282786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.282786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.282997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.283118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] use_journal = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.283233 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.283352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] use_json = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.283473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] use_journal = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.283587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] use_stderr = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.283706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] use_json = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.283829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] use_syslog = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.283949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] use_stderr = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.283949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] use_syslog = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.284173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.284286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:25:32.284400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.284522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.284638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.284751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.284864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.284977 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.285090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.285214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.285335 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.285450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.285570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.285684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.285798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.285943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.286080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.286206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.286327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.286438 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.286557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.286680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.286795 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.286908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.287021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.287140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.287254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.287375 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.287490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.287604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.287724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.287844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.287958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.288152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.288269 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.288382 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.288496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.288609 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.288724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.288845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.288959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.289072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.289192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.289305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.289428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.289803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.289935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.290051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.290165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.290278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.290391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.290516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.290633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.290746 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.290860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.290973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.291098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.291214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.291332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.291453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.291630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.291747 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.291870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.291985 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.292099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.292212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.292325 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.292444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.293065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.293180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.293293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.293406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.293526 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.293640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.293760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.293880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.294013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.294127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.294246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.294369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.294560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.294677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.294796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.294910 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.295030 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.295144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.295269 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.295383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.295502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.295616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.295735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.295855 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.295976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.296091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.296205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.296317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.296415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.296516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.296626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.296723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.296817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.296918 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.297015 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.297110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.297216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.297312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.297416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.297529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.297652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.297797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.298001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.298115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.298230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.298344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.298449 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.298562 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.298688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.298805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.298925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.299039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.299160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.299282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.299388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.299510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.299626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.299748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.299870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.299988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.300115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.300231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.300353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.300469 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.300573 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.300686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.300800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.300925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.301040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.301155 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.301259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.301372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.301493 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.301605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.301726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.301841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.301978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.302096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.302212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.302338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.302455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.302569 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.302683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.302797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.302912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.303033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.303148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.303262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.303377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.303491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.303606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.303727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.303849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.303964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.304079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.304196 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.304311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.304437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.304553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.304667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.304781 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.304896 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.305010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.305131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.305247 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.305362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.305478 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.305598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.305714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.305903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.306036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.306151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.306266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.306380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.306496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.306626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.306752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.306879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.307002 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.307118 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.307233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.307348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.307463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.307586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.307703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.307818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.307933 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.308054 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.308729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.308849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.308965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.309080 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.309195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.309316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.309440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.309558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.309674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.309789 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.309904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.310064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.310192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.310310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.310426 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.external = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.310541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] auth.external = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.310656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.310778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.310908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.311051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.311182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.311301 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.311417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.password = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.311533 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] auth.password = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.311649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] auth.token = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.311766 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] auth.token = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.311889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.312021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.312137 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.312253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.312377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.312500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.312625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.312742 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.312859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.312975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.313091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.313207 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.313330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.313703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.313827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.313958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] credential.caching = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.314077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.314197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.314326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.314445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.314568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.314685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.314801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.314917 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.315041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.315158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.315274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.315396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.315513 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.315629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.315756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.315881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.316000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.316116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.316232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.316347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.316475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.316593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.316709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.316824 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.316940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.317056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.317180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.317297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.317413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.317529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.317645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.317760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] federation.caching = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.317884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.318021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.318138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.318253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.318368 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.318484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.318611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.318729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.318845 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.318961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.319079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.319195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.319319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.319444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.319561 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.319677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.319792 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.319916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.320033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.320151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.320266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.320381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.320497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.320687 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.caching = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.320807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.320923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.321046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.321162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.321279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.321403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.321520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.321636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.321752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.321752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.321986 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.322114 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.322234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.322234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.322451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.322566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.322681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.322681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.322907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.323023 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.323138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.323253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.323370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.323370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.323596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.323712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.323828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.323944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.324059 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.324187 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.324304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.324304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.324530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.324530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.324747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.324747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.324973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.325089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.325204 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.325320 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.325437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.325567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.325685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.325800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.325934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.326052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.326167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.326299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.326417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.326533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.326649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.326764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.326880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.327003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.327119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.327235 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.327350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.327465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.327589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.327716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.327833 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.327949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.328065 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.328179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.328295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.328418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.328535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.328651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.328766 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.328880 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.328996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.329119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.329236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.329361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.329477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.329594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.329709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.329832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.329965 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.330081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.330197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.330312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.330428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.330560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.330686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.330803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.330918 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.331033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.331148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.331272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.331389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.331505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.331621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.331738 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.331861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.331978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.332094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.332210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.332325 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.332441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.332564 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.332681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.332798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.332913 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.333029 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.333144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.333267 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.333384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.333500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.333615 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.333730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.333847 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.334021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.334141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.334260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.334376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.334491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.334606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.334729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.334848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.334963 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.335079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.335193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.335309 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.335431 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.335554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.335671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.335671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.335889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.336004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.336128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.336245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.336361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.336361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.336577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.336693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.336816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.336935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.337051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.337166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.337282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.337397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.337520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.337637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.337752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.337868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.337993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.338109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.338235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.338353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.338468 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.338584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.338699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.338814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.338938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.339056 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.339171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.339286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.339401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.339516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.339640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.339756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.339872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.339987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.340104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.340219 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.340343 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.340460 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.340575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.340691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.340807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.340922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.341047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.341164 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.341280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.341396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.341512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] resource.caching = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.341627 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.341751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.341867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.342036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.342155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.342271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.342387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.342515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.342633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.342748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.342865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.342980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.343095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.343285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.343404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.343521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.343637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] role.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.343753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.343869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] role.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.343993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] role.caching = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.344110 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.344229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] role.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.344345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.344460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.344576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.344701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.344818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.344934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.345050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.345166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.345290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.345408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.345524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.345639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.345755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.345870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.346017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.346137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.346253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.346370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.346486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.346602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.346725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.346842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.346958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.347074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.347190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.347305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.347305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.347533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.347533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.347750 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.347865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.347980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.348103 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.348220 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.348336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.348453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.348569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.348685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.348809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.348926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.349042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.349158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.349274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.349397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.349515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.349631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.349746 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.349862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.349992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.350119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.350237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.350353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.350468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.350585 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.350700 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.350824 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.350941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.351057 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.351173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.351289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.351404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.351528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.351663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.351781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.351897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.352014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.352129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.352253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] token.caching = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.352371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.352487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.352602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.352719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.352834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.352958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.353075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.353191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.353306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.353422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.353538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.353662 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.353780 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.353895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.354067 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.354433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.354551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.354678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.354798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.355415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.355533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.355649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.355765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.355890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.356007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.356123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.356239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.356354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.356470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.356594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.356594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.356812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.356812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.connection = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.357038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.357164 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.357281 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.connection = **** {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.357409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.357534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.357659 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.357776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.357893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.358028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.358144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.358259 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.358387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.358504 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.358620 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.358736 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.358852 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.358969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.359092 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.359210 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.359326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.359442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.359558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.359673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.359797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.359914 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.360031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.360137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.360251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.360367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.360490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.360607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.360724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.360840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.360955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.361071 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.361195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.361312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.361428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 12:25:32.361545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:25:32.361545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80326) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 12:25:32.386372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.386598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.386821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.387315 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.387543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.387762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.395070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.396017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.424799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.425022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.425600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.425831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.476569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.476780 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.527538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.527762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.528029 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.528262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.591618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.591783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.591871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.592031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.592121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.592251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.592331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.592433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.592752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.592752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.592902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.592902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.616075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.616302 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.616431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.616514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.616588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.616723 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.616806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.616940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.617023 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.617150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.617231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.617361 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.617442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.617575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.628462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.628835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.641530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.641628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.641762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.641942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.642025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.642100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.642219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.642314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.642436 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.642519 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.642649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.642739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.642862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80326) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.642945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:25:32.660428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.660635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 12:25:32.660920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.661004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.661129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 12:25:32.661212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 12:25:32.661401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.661482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.661643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 12:25:32.661732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.661897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.662005 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.662173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.662253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.662425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.662680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.673513 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.673786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.673977 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.674086 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.674257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.674364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.674523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.674799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.682551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.682952 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.685816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.686114 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.686235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.686389 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.686509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.686663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.686806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.686937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.687041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.687192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.687314 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.687454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.687567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.687719 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.687830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.687981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.688097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.688246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.688360 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.688503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.688625 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.688883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.716657 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.717223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.718274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.718551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.718846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.719121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.723755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.724145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.724320 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.724423 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.724714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.724994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.732519 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.732751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.732992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.733270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.737172 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.737456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.737712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.737796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.738018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.738104 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.738289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.738368 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.738558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.738650 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.738820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.738900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.739077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.739157 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.739335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.739593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.757604 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.757874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.758003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.758273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.762456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.762796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.766079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.766439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.766521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.766834 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.771417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.771796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.775030 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.775406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.779197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.779497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.779579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.779774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.779854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.780050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.780132 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.780323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.780405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.780605 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.780695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.780921 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.800091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.800370 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.800631 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.800898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.802097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.802381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.802643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.802916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.808767 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.810470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.810743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.810859 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.812458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.812729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.813883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.814185 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.814466 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.814743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.815013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.815889 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.816165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.816443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.816717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.816987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.828257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.828545 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.828820 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.829096 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.829462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.829734 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.830034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.830265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.830540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.830814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.831088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.831454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.831736 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.832008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.848158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.850116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.851402 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.853318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.855539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.857270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.857425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.859003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.859177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.859284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.859560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.860805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.861079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.861354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.866857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.868462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.868584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.870426 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.871692 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.871975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.872355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.872642 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.873654 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.873955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.874343 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.874632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.881584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.881868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.882167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.883571 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.883852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.884128 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.890508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.890782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.891055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.891325 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.891593 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.891955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.892233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.892432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80326) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:25:32.892522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.892703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.892785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.892972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.893053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.893237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.893320 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.893502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.893591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.893794 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.893876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.894075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.894157 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.894421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.894687 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.894956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.895221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.895668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.895758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.896001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80326) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:25:32.931775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55de56ee09c0 pid: 80327 (default app) Jul 25 12:25:32.933557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55de56ee09c0 pid: 80326 (default app) Jul 25 12:25:32.935188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbf2b9-e565-405d-8ea0-bd2d1c7dc9ff None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:32.935479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbf2b9-e565-405d-8ea0-bd2d1c7dc9ff None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:32.935752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbf2b9-e565-405d-8ea0-bd2d1c7dc9ff None None] PATH_INFO: `/v3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:32.939960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1/1] 10.222.0.60 () {50 vars in 791 bytes} [Mon Jul 25 12:25:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:40.486450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88964ac6-73ad-4da7-9508-30d0905fc455 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:40.486970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88964ac6-73ad-4da7-9508-30d0905fc455 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:40.486970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88964ac6-73ad-4da7-9508-30d0905fc455 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:40.491323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1/2] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:25:40.497061 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cd0af-73ef-42f3-a208-7aacdad37c3f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:40.497472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cd0af-73ef-42f3-a208-7aacdad37c3f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:40.497612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cd0af-73ef-42f3-a208-7aacdad37c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:40.567485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: INFO dbcounter [None req-d82cd0af-73ef-42f3-a208-7aacdad37c3f None None] Registered counter for database keystone Jul 25 12:25:40.588165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG oslo_db.sqlalchemy.engines [None req-d82cd0af-73ef-42f3-a208-7aacdad37c3f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80327) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 12:25:40.594271 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writer thread running {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 12:25:40.767418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG passlib.handlers.bcrypt [None req-d82cd0af-73ef-42f3-a208-7aacdad37c3f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80327) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 12:25:40.767753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG passlib.handlers.bcrypt [None req-d82cd0af-73ef-42f3-a208-7aacdad37c3f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80327) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 12:25:40.822108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d82cd0af-73ef-42f3-a208-7aacdad37c3f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:40.824230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d82cd0af-73ef-42f3-a208-7aacdad37c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:40.947358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 2/3] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 12:25:41.073614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88964ac6-73ad-4da7-9508-30d0905fc455 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:41.143556 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: INFO dbcounter [None req-88964ac6-73ad-4da7-9508-30d0905fc455 None None] Registered counter for database keystone Jul 25 12:25:41.163142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG oslo_db.sqlalchemy.engines [None req-88964ac6-73ad-4da7-9508-30d0905fc455 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80326) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 12:25:41.169710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writer thread running {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 12:25:41.287581 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88964ac6-73ad-4da7-9508-30d0905fc455 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:41.291865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13c5eb-d5b1-4220-88dd-0d6b908de527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:41.292613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13c5eb-d5b1-4220-88dd-0d6b908de527 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:41.292900 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13c5eb-d5b1-4220-88dd-0d6b908de527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:41.293615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13c5eb-d5b1-4220-88dd-0d6b908de527 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:41.301633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13c5eb-d5b1-4220-88dd-0d6b908de527 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:41.351775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4a13c5eb-d5b1-4220-88dd-0d6b908de527 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 12:25:41.352063 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13c5eb-d5b1-4220-88dd-0d6b908de527 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:41.361875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 2/4] 10.222.0.60 () {58 vars in 1324 bytes} [Mon Jul 25 12:25:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 12:25:41.367834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cd0af-73ef-42f3-a208-7aacdad37c3f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:41.382316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cd0af-73ef-42f3-a208-7aacdad37c3f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:41.387180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94e4fd-a8fb-40f8-8ddc-6b78433fc51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:41.388290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94e4fd-a8fb-40f8-8ddc-6b78433fc51c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:41.388576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94e4fd-a8fb-40f8-8ddc-6b78433fc51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:41.388849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94e4fd-a8fb-40f8-8ddc-6b78433fc51c admin admin] PATH_INFO: `/v3/projects/61157336119141239b389248fadfbdf0` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:41.390361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94e4fd-a8fb-40f8-8ddc-6b78433fc51c admin admin] RBAC: Authorizing `identity:get_project(project_id=61157336119141239b389248fadfbdf0)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:41.442157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cc94e4fd-a8fb-40f8-8ddc-6b78433fc51c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 12:25:41.442499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94e4fd-a8fb-40f8-8ddc-6b78433fc51c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:41.447043 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 3/5] 10.222.0.60 () {56 vars in 1207 bytes} [Mon Jul 25 12:25:41 2022] GET /identity/v3/projects/61157336119141239b389248fadfbdf0 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:43.399586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f59ca-097a-4f8d-a621-3117291686c4 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:43.400114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f59ca-097a-4f8d-a621-3117291686c4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:43.400114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f59ca-097a-4f8d-a621-3117291686c4 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:43.404301 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 3/6] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:25:43.409629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f11f1b-5c0d-4ff8-8b74-0698ff0de227 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:43.410075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f11f1b-5c0d-4ff8-8b74-0698ff0de227 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:43.410226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f11f1b-5c0d-4ff8-8b74-0698ff0de227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:43.430615 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-49f11f1b-5c0d-4ff8-8b74-0698ff0de227 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:43.432128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-49f11f1b-5c0d-4ff8-8b74-0698ff0de227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:43.440688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 4/7] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 12:25:43.461936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f59ca-097a-4f8d-a621-3117291686c4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:43.473788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f59ca-097a-4f8d-a621-3117291686c4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:43.477981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b33a00-1b50-4534-97f9-1b1988761706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:43.478614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b33a00-1b50-4534-97f9-1b1988761706 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:43.478899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b33a00-1b50-4534-97f9-1b1988761706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:43.479179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b33a00-1b50-4534-97f9-1b1988761706 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:43.490286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b33a00-1b50-4534-97f9-1b1988761706 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:43.492061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b33a00-1b50-4534-97f9-1b1988761706 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:43.501956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 4/8] 10.222.0.60 () {58 vars in 1324 bytes} [Mon Jul 25 12:25:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:25:43.507483 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f11f1b-5c0d-4ff8-8b74-0698ff0de227 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:43.518123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f11f1b-5c0d-4ff8-8b74-0698ff0de227 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:43.522458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a998d4-52ff-4931-a893-e57a68ce3852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:43.523322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a998d4-52ff-4931-a893-e57a68ce3852 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:43.523603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a998d4-52ff-4931-a893-e57a68ce3852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:43.523877 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a998d4-52ff-4931-a893-e57a68ce3852 admin admin] PATH_INFO: `/v3/users/6bca01c7266944329ecb3cbe0887bc8e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:43.525264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a998d4-52ff-4931-a893-e57a68ce3852 admin admin] RBAC: Authorizing `identity:get_user(user_id=6bca01c7266944329ecb3cbe0887bc8e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:43.527162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a998d4-52ff-4931-a893-e57a68ce3852 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:43.528730 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 5/9] 10.222.0.60 () {56 vars in 1198 bytes} [Mon Jul 25 12:25:43 2022] GET /identity/v3/users/6bca01c7266944329ecb3cbe0887bc8e => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:45.583052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaec415-73d3-4962-8da0-e8c480af4828 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:45.583552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaec415-73d3-4962-8da0-e8c480af4828 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:45.583649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaec415-73d3-4962-8da0-e8c480af4828 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:45.584964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 5/10] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:25:45.590151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a20e0-1c2e-43b4-a9c1-ea47376469a4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:45.590541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a20e0-1c2e-43b4-a9c1-ea47376469a4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:45.590711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a20e0-1c2e-43b4-a9c1-ea47376469a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:45.608408 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-518a20e0-1c2e-43b4-a9c1-ea47376469a4 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:45.609583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-518a20e0-1c2e-43b4-a9c1-ea47376469a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:45.614866 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650665cc-bd17-4eb0-8a50-ab8f30ef372d None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:45.615159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650665cc-bd17-4eb0-8a50-ab8f30ef372d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:45.615443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650665cc-bd17-4eb0-8a50-ab8f30ef372d None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:45.616778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 6/11] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:25:45.622224 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9799eb-f89c-4e7e-88de-d98107e8e9b2 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:45.622611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9799eb-f89c-4e7e-88de-d98107e8e9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:45.622899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9799eb-f89c-4e7e-88de-d98107e8e9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:45.665382 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG passlib.handlers.bcrypt [None req-fc9799eb-f89c-4e7e-88de-d98107e8e9b2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80326) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 12:25:45.665807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG passlib.handlers.bcrypt [None req-fc9799eb-f89c-4e7e-88de-d98107e8e9b2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80326) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 12:25:45.690240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 6/12] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:25:45.711820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-fc9799eb-f89c-4e7e-88de-d98107e8e9b2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:45.713336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a20e0-1c2e-43b4-a9c1-ea47376469a4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:45.713490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fc9799eb-f89c-4e7e-88de-d98107e8e9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:45.737657 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:45.812639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 7/13] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:25:45.817790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a20e0-1c2e-43b4-a9c1-ea47376469a4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:45.833540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9799eb-f89c-4e7e-88de-d98107e8e9b2 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:45.851040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1d66b-ad40-4e14-998f-ad730e4e3899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:45.851836 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1d66b-ad40-4e14-998f-ad730e4e3899 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:45.852123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1d66b-ad40-4e14-998f-ad730e4e3899 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:45.852404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1d66b-ad40-4e14-998f-ad730e4e3899 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:45.864214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1d66b-ad40-4e14-998f-ad730e4e3899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b9c1d66b-ad40-4e14-998f-ad730e4e3899 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:25:45.875922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:25:45.877038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:25:45.878016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 7/14] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:25:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:25:45.883112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1d66b-ad40-4e14-998f-ad730e4e3899 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:45.934280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9799eb-f89c-4e7e-88de-d98107e8e9b2 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:45.936536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:45.964426 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:45.971413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fafa5b-c16b-485a-ab8c-5a5b562bc165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:45.972315 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fafa5b-c16b-485a-ab8c-5a5b562bc165 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:45.972603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fafa5b-c16b-485a-ab8c-5a5b562bc165 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:45.972880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fafa5b-c16b-485a-ab8c-5a5b562bc165 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:45.984485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1d66b-ad40-4e14-998f-ad730e4e3899 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:46.018127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9980491-e869-45e8-810f-5df7b82b752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:46.018795 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9980491-e869-45e8-810f-5df7b82b752b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:46.019079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9980491-e869-45e8-810f-5df7b82b752b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:46.019358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9980491-e869-45e8-810f-5df7b82b752b None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:46.020108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9980491-e869-45e8-810f-5df7b82b752b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:25:46.020414 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9980491-e869-45e8-810f-5df7b82b752b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:46.021816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9980491-e869-45e8-810f-5df7b82b752b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:46.034345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 8/15] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:25:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:25:46.039884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9980491-e869-45e8-810f-5df7b82b752b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:46.042443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fafa5b-c16b-485a-ab8c-5a5b562bc165 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:46.044485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fafa5b-c16b-485a-ab8c-5a5b562bc165 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:46.137547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9980491-e869-45e8-810f-5df7b82b752b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:46.139156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 8/16] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:25:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:25:46.144710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fafa5b-c16b-485a-ab8c-5a5b562bc165 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:46.147623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:46.172420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe55749-fd50-4fff-b984-18f67c3aa50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:46.173296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe55749-fd50-4fff-b984-18f67c3aa50e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:46.173578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe55749-fd50-4fff-b984-18f67c3aa50e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:46.173853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe55749-fd50-4fff-b984-18f67c3aa50e None admin] PATH_INFO: `/v3/users/6bca01c7266944329ecb3cbe0887bc8e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:46.175070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe55749-fd50-4fff-b984-18f67c3aa50e None admin] RBAC: Authorizing `identity:get_user(user_id=6bca01c7266944329ecb3cbe0887bc8e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:46.176534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe55749-fd50-4fff-b984-18f67c3aa50e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:46.178383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 9/17] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:25:46 2022] GET /identity/v3/users/6bca01c7266944329ecb3cbe0887bc8e => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:25:46.183752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe55749-fd50-4fff-b984-18f67c3aa50e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:46.198482 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:46.246234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fafa5b-c16b-485a-ab8c-5a5b562bc165 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:46.279336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df567f5-bc9c-4f0e-a651-372d03a61583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:46.280128 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df567f5-bc9c-4f0e-a651-372d03a61583 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:46.280414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df567f5-bc9c-4f0e-a651-372d03a61583 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:46.280692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df567f5-bc9c-4f0e-a651-372d03a61583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:46.284280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe55749-fd50-4fff-b984-18f67c3aa50e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:46.299818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df567f5-bc9c-4f0e-a651-372d03a61583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:46.301288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df567f5-bc9c-4f0e-a651-372d03a61583 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:46.317066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f256c2-9521-48af-9ed5-74e18b680ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:46.317851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f256c2-9521-48af-9ed5-74e18b680ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:46.318180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f256c2-9521-48af-9ed5-74e18b680ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:46.318456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f256c2-9521-48af-9ed5-74e18b680ac7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:46.319621 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f256c2-9521-48af-9ed5-74e18b680ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:46.321058 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f256c2-9521-48af-9ed5-74e18b680ac7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-6df567f5-bc9c-4f0e-a651-372d03a61583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:25:46.321575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:25:46.322395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:25:46.323191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:25:46.323191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:25:46.323191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:25:46.323191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:25:46.323191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 9/18] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:25:46.323456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 10/19] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:25:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:25:46.328448 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df567f5-bc9c-4f0e-a651-372d03a61583 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:46.329502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f256c2-9521-48af-9ed5-74e18b680ac7 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:46.383288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:46.429151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df567f5-bc9c-4f0e-a651-372d03a61583 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:46.465274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a093a0-8665-459e-83ec-5939bfadc4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:46.465964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a093a0-8665-459e-83ec-5939bfadc4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:46.466250 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a093a0-8665-459e-83ec-5939bfadc4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:46.466528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a093a0-8665-459e-83ec-5939bfadc4f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:46.467382 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a093a0-8665-459e-83ec-5939bfadc4f2 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.domain.id=default, user.id=6bca01c7266944329ecb3cbe0887bc8e) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:25:46.467695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a093a0-8665-459e-83ec-5939bfadc4f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:46.469192 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a093a0-8665-459e-83ec-5939bfadc4f2 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:46.478873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 10/20] 10.222.0.60 () {56 vars in 1322 bytes} [Mon Jul 25 12:25:46 2022] GET /identity/v3/role_assignments?user.id=6bca01c7266944329ecb3cbe0887bc8e&scope.domain.id=default&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:25:46.517539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:46.538983 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f256c2-9521-48af-9ed5-74e18b680ac7 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:46.574361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d33411-0a7b-4fa6-ae1c-36271170e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:46.575037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d33411-0a7b-4fa6-ae1c-36271170e13a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:46.575319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d33411-0a7b-4fa6-ae1c-36271170e13a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:46.575597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d33411-0a7b-4fa6-ae1c-36271170e13a None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:46.576337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d33411-0a7b-4fa6-ae1c-36271170e13a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:25:46.576642 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d33411-0a7b-4fa6-ae1c-36271170e13a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:46.578082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d33411-0a7b-4fa6-ae1c-36271170e13a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:46.597052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 11/21] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:25:48.888987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d90b2-bdf0-4988-ad95-81d39ad1f4c4 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:48.889499 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d90b2-bdf0-4988-ad95-81d39ad1f4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:48.889608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d90b2-bdf0-4988-ad95-81d39ad1f4c4 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:48.890948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 11/22] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:25:48.896379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581d871-08af-4db8-b74f-f9a9c18da83c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:48.896811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581d871-08af-4db8-b74f-f9a9c18da83c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:48.897118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581d871-08af-4db8-b74f-f9a9c18da83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:48.922891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1581d871-08af-4db8-b74f-f9a9c18da83c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:48.924094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1581d871-08af-4db8-b74f-f9a9c18da83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:49.003926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826ac92-edb9-4810-9959-8c16b6ed36a9 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:49.004260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826ac92-edb9-4810-9959-8c16b6ed36a9 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:49.004492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826ac92-edb9-4810-9959-8c16b6ed36a9 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:49.006044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 12/23] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:25:49.010771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775878-faed-4ab8-8bba-c80158f872fb None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:49.011067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775878-faed-4ab8-8bba-c80158f872fb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:49.011916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775878-faed-4ab8-8bba-c80158f872fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:49.019390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 12/24] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:25:49.055206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-16775878-faed-4ab8-8bba-c80158f872fb None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:49.057309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-16775878-faed-4ab8-8bba-c80158f872fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:49.062419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581d871-08af-4db8-b74f-f9a9c18da83c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:49.180583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:49.231327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 13/25] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:25:49.232555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581d871-08af-4db8-b74f-f9a9c18da83c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:49.249177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e87bb3-55a0-4c0a-9a80-8da0df0dfc6a None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:49.249675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e87bb3-55a0-4c0a-9a80-8da0df0dfc6a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:49.250145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e87bb3-55a0-4c0a-9a80-8da0df0dfc6a None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:49.251660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 14/26] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:25:49.256613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931afd2-4b66-496e-b524-39c89695f146 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:49.256905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931afd2-4b66-496e-b524-39c89695f146 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:49.257183 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931afd2-4b66-496e-b524-39c89695f146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:49.284680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc30767-3637-4b34-b25a-bfd56003f4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:49.285480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc30767-3637-4b34-b25a-bfd56003f4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:49.285762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc30767-3637-4b34-b25a-bfd56003f4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:49.286073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc30767-3637-4b34-b25a-bfd56003f4d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:49.297140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc30767-3637-4b34-b25a-bfd56003f4d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-afc30767-3637-4b34-b25a-bfd56003f4d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:25:49.304850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:25:49.305897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:25:49.306925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:25:49.306925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:25:49.306925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:25:49.306925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:25:49.306925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:25:49.306925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:25:49.306925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:25:49.306925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:25:49.306925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:25:49.306925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:25:49.306925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:25:49.306925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:25:49.308266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 13/27] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:25:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:25:49.309091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc30767-3637-4b34-b25a-bfd56003f4d4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:49.309319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-c931afd2-4b66-496e-b524-39c89695f146 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:49.310564 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c931afd2-4b66-496e-b524-39c89695f146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:49.411320 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc30767-3637-4b34-b25a-bfd56003f4d4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:49.446308 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:49.455358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:49.456878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b9530-8b1a-49d4-a62d-94bbb652713e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:49.457736 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b9530-8b1a-49d4-a62d-94bbb652713e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:49.458055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b9530-8b1a-49d4-a62d-94bbb652713e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:49.459229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b9530-8b1a-49d4-a62d-94bbb652713e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:49.479653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b9530-8b1a-49d4-a62d-94bbb652713e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:49.481451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b9530-8b1a-49d4-a62d-94bbb652713e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:49.502354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 15/28] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:25:49.503264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931afd2-4b66-496e-b524-39c89695f146 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-f42b9530-8b1a-49d4-a62d-94bbb652713e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:25:49.519906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:25:49.520774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:25:49.521567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:25:49.521567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:25:49.521567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:25:49.521567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:25:49.521567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 14/29] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:25:49.528204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b9530-8b1a-49d4-a62d-94bbb652713e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:49.666221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b9530-8b1a-49d4-a62d-94bbb652713e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:49.702502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658278b-bda5-4fe8-b1a9-9bf1f3bd2e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:49.703094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658278b-bda5-4fe8-b1a9-9bf1f3bd2e7a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:49.703376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658278b-bda5-4fe8-b1a9-9bf1f3bd2e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:49.703655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658278b-bda5-4fe8-b1a9-9bf1f3bd2e7a None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:49.704048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931afd2-4b66-496e-b524-39c89695f146 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:49.704394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7658278b-bda5-4fe8-b1a9-9bf1f3bd2e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:25:49.704703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7658278b-bda5-4fe8-b1a9-9bf1f3bd2e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:49.706250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7658278b-bda5-4fe8-b1a9-9bf1f3bd2e7a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:49.719598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 15/30] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:49.720466 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658278b-bda5-4fe8-b1a9-9bf1f3bd2e7a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:49.762329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d996254-b93e-4cf2-8146-2ef9e37f6a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:49.763003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d996254-b93e-4cf2-8146-2ef9e37f6a75 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:49.763282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d996254-b93e-4cf2-8146-2ef9e37f6a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:49.763553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d996254-b93e-4cf2-8146-2ef9e37f6a75 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:49.764404 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d996254-b93e-4cf2-8146-2ef9e37f6a75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:25:49.764709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d996254-b93e-4cf2-8146-2ef9e37f6a75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:49.766222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d996254-b93e-4cf2-8146-2ef9e37f6a75 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:49.767369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:49.779568 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 16/31] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:25:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:49.780389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d996254-b93e-4cf2-8146-2ef9e37f6a75 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:49.851474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:49.862922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658278b-bda5-4fe8-b1a9-9bf1f3bd2e7a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:49.899101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9e90f-414c-496b-9280-ab320948a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:49.899879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e90f-414c-496b-9280-ab320948a915 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:49.900063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e90f-414c-496b-9280-ab320948a915 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:49.900345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e90f-414c-496b-9280-ab320948a915 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:49.901109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9e90f-414c-496b-9280-ab320948a915 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:49.902764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9e90f-414c-496b-9280-ab320948a915 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:49.967992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d996254-b93e-4cf2-8146-2ef9e37f6a75 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:50.026555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d2029-0a88-4c3c-b410-db864c8e8ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:50.027257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d2029-0a88-4c3c-b410-db864c8e8ff4 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:50.027537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d2029-0a88-4c3c-b410-db864c8e8ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:50.027811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d2029-0a88-4c3c-b410-db864c8e8ff4 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:50.028761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d2029-0a88-4c3c-b410-db864c8e8ff4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:50.030244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d2029-0a88-4c3c-b410-db864c8e8ff4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:50.033426 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 16/32] 10.222.0.60 () {60 vars in 1131 bytes} [Mon Jul 25 12:25:49 2022] POST /identity/v3/roles => generated 219 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:25:50.039113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9e90f-414c-496b-9280-ab320948a915 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:50.175821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9e90f-414c-496b-9280-ab320948a915 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:50.177793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:50.223004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bd159-c891-4232-8ee6-da4a2c1a4c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:50.223837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd159-c891-4232-8ee6-da4a2c1a4c76 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:50.224128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd159-c891-4232-8ee6-da4a2c1a4c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:50.224411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd159-c891-4232-8ee6-da4a2c1a4c76 None admin] PATH_INFO: `/v3/users/6bca01c7266944329ecb3cbe0887bc8e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:50.225768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bd159-c891-4232-8ee6-da4a2c1a4c76 None admin] RBAC: Authorizing `identity:get_user(user_id=6bca01c7266944329ecb3cbe0887bc8e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:50.227462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bd159-c891-4232-8ee6-da4a2c1a4c76 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:50.229129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 17/33] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:25:50 2022] GET /identity/v3/users/6bca01c7266944329ecb3cbe0887bc8e => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:50.232366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca0f9b-8812-4155-a015-cec05039ca36 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:50.232659 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca0f9b-8812-4155-a015-cec05039ca36 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:50.232942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca0f9b-8812-4155-a015-cec05039ca36 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:50.234218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 18/34] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:25:50.237840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808de4ad-0bef-46db-955a-fb105efb842e None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:50.238163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808de4ad-0bef-46db-955a-fb105efb842e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:50.238450 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808de4ad-0bef-46db-955a-fb105efb842e None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:50.240039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 19/35] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:25:50.241027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555157c4-b4ff-4894-8cee-41e87cbb1329 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:50.241314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555157c4-b4ff-4894-8cee-41e87cbb1329 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:50.241592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555157c4-b4ff-4894-8cee-41e87cbb1329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:50.265702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-555157c4-b4ff-4894-8cee-41e87cbb1329 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:50.267260 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-555157c4-b4ff-4894-8cee-41e87cbb1329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:50.301877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 17/36] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:25:49 2022] POST /identity/v3/projects => generated 301 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:25:50.303395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0487ab6-f51b-4f93-927a-49f7b8929b06 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:50.303686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0487ab6-f51b-4f93-927a-49f7b8929b06 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:50.303966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0487ab6-f51b-4f93-927a-49f7b8929b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:50.331317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-a0487ab6-f51b-4f93-927a-49f7b8929b06 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:50.333012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a0487ab6-f51b-4f93-927a-49f7b8929b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:50.360713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 20/37] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:25:50.371639 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555157c4-b4ff-4894-8cee-41e87cbb1329 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:50.423639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 18/38] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:25:50.425010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0487ab6-f51b-4f93-927a-49f7b8929b06 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:50.464444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:50.515369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:50.518565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555157c4-b4ff-4894-8cee-41e87cbb1329 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:50.552640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0487ab6-f51b-4f93-927a-49f7b8929b06 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:50.559358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0036a2-75d8-4e13-9548-53db0ee8b27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:50.560158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0036a2-75d8-4e13-9548-53db0ee8b27e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:50.560444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0036a2-75d8-4e13-9548-53db0ee8b27e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:50.560725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0036a2-75d8-4e13-9548-53db0ee8b27e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:50.561943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0036a2-75d8-4e13-9548-53db0ee8b27e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:50.563392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0036a2-75d8-4e13-9548-53db0ee8b27e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:50.565016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 21/39] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:25:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:50.565754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0036a2-75d8-4e13-9548-53db0ee8b27e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:50.590585 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5419ae-07f7-4e3f-b885-b45c33f6a68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:50.591256 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5419ae-07f7-4e3f-b885-b45c33f6a68e None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:50.591537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5419ae-07f7-4e3f-b885-b45c33f6a68e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:50.591813 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5419ae-07f7-4e3f-b885-b45c33f6a68e None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:50.592556 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5419ae-07f7-4e3f-b885-b45c33f6a68e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:50.594086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5419ae-07f7-4e3f-b885-b45c33f6a68e None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:50.610328 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 19/40] 10.222.0.60 () {60 vars in 1131 bytes} [Mon Jul 25 12:25:50 2022] POST /identity/v3/roles => generated 223 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:25:50.611593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2270f4b-fc0a-4083-b2f9-4cae29afa42f None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:50.611882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2270f4b-fc0a-4083-b2f9-4cae29afa42f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:50.612161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2270f4b-fc0a-4083-b2f9-4cae29afa42f None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:50.613420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 20/41] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:25:50.617719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2270f4b-fc0a-4083-b2f9-4cae29afa42f None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:50.684696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0036a2-75d8-4e13-9548-53db0ee8b27e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:50.718060 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:50.724428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec99ae0-f23b-4a5f-b773-d84c77e896e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:50.725279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec99ae0-f23b-4a5f-b773-d84c77e896e6 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:50.725572 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec99ae0-f23b-4a5f-b773-d84c77e896e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:50.726157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec99ae0-f23b-4a5f-b773-d84c77e896e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:50.727106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec99ae0-f23b-4a5f-b773-d84c77e896e6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:50.728731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec99ae0-f23b-4a5f-b773-d84c77e896e6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:50.739565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2270f4b-fc0a-4083-b2f9-4cae29afa42f None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:50.747524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 22/42] 10.222.0.60 () {60 vars in 1131 bytes} [Mon Jul 25 12:25:50 2022] POST /identity/v3/roles => generated 225 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:25:50.760028 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e295c2-dd44-478b-a6c5-f571d0e9836f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:50.760028 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e295c2-dd44-478b-a6c5-f571d0e9836f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:50.760028 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e295c2-dd44-478b-a6c5-f571d0e9836f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:50.780213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-99e295c2-dd44-478b-a6c5-f571d0e9836f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:50.781717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-99e295c2-dd44-478b-a6c5-f571d0e9836f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:50.785733 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675892c-0cd0-4bd8-93d9-e7424c82ba73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:50.786724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675892c-0cd0-4bd8-93d9-e7424c82ba73 None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:50.787023 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675892c-0cd0-4bd8-93d9-e7424c82ba73 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:50.787308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675892c-0cd0-4bd8-93d9-e7424c82ba73 None admin] PATH_INFO: `/v3/domains/default/users/6bca01c7266944329ecb3cbe0887bc8e/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:50.789676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675892c-0cd0-4bd8-93d9-e7424c82ba73 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6bca01c7266944329ecb3cbe0887bc8e, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:50.792746 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675892c-0cd0-4bd8-93d9-e7424c82ba73 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:50.811936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:50.813724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 21/43] 10.222.0.60 () {58 vars in 1361 bytes} [Mon Jul 25 12:25:50 2022] PUT /identity/v3/domains/default/users/6bca01c7266944329ecb3cbe0887bc8e/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:25:50.878398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 23/44] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:25:50.901850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675892c-0cd0-4bd8-93d9-e7424c82ba73 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:51.012907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675892c-0cd0-4bd8-93d9-e7424c82ba73 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:51.046095 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab140b5-2aee-4525-aed8-d1554461085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:51.046895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab140b5-2aee-4525-aed8-d1554461085b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:51.047182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab140b5-2aee-4525-aed8-d1554461085b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:51.047461 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab140b5-2aee-4525-aed8-d1554461085b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:51.048644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab140b5-2aee-4525-aed8-d1554461085b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:51.050141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab140b5-2aee-4525-aed8-d1554461085b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:51.051769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 22/45] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:25:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:51.057396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e295c2-dd44-478b-a6c5-f571d0e9836f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:51.157949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e295c2-dd44-478b-a6c5-f571d0e9836f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:51.192774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385a772-a361-448b-970e-3b9b01d0ba0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:51.193441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385a772-a361-448b-970e-3b9b01d0ba0c None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:51.193727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385a772-a361-448b-970e-3b9b01d0ba0c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:51.194034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385a772-a361-448b-970e-3b9b01d0ba0c None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:51.194747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385a772-a361-448b-970e-3b9b01d0ba0c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:51.196209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385a772-a361-448b-970e-3b9b01d0ba0c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:51.222990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 24/46] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:25:51 2022] POST /identity/v3/projects => generated 312 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:25:52.795041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1bb00-f464-4e6c-a7f2-865e12f5a276 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:52.795548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1bb00-f464-4e6c-a7f2-865e12f5a276 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:52.795629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1bb00-f464-4e6c-a7f2-865e12f5a276 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:52.796948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 23/47] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:25:52.802191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a0955-7fe7-46f3-871d-1eafd11d20d5 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:52.802577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a0955-7fe7-46f3-871d-1eafd11d20d5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:52.802756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a0955-7fe7-46f3-871d-1eafd11d20d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:52.820388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-303a0955-7fe7-46f3-871d-1eafd11d20d5 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:52.821573 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-303a0955-7fe7-46f3-871d-1eafd11d20d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:52.824724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:52.887868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 25/48] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:25:52.911797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1bb00-f464-4e6c-a7f2-865e12f5a276 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:52.927139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:53.012859 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1bb00-f464-4e6c-a7f2-865e12f5a276 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:53.045978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3a0e7-d8f3-41e7-9583-613d2c2f2617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:53.046710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3a0e7-d8f3-41e7-9583-613d2c2f2617 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:53.046994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3a0e7-d8f3-41e7-9583-613d2c2f2617 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:53.047269 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3a0e7-d8f3-41e7-9583-613d2c2f2617 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:53.058197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3a0e7-d8f3-41e7-9583-613d2c2f2617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-3aa3a0e7-d8f3-41e7-9583-613d2c2f2617 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:25:53.066805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:25:53.067801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:25:53.068884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 24/49] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:25:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:25:53.073745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a0955-7fe7-46f3-871d-1eafd11d20d5 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:53.172702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a0955-7fe7-46f3-871d-1eafd11d20d5 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:53.208131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6609ae-1ab6-4cdd-a9f6-b05c6c3b3567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:53.208797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6609ae-1ab6-4cdd-a9f6-b05c6c3b3567 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:53.209078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6609ae-1ab6-4cdd-a9f6-b05c6c3b3567 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:53.209354 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6609ae-1ab6-4cdd-a9f6-b05c6c3b3567 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:53.210128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6609ae-1ab6-4cdd-a9f6-b05c6c3b3567 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:25:53.210434 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6609ae-1ab6-4cdd-a9f6-b05c6c3b3567 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:53.211811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6609ae-1ab6-4cdd-a9f6-b05c6c3b3567 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:53.220365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 26/50] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:53.225888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3a0e7-d8f3-41e7-9583-613d2c2f2617 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:53.229680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7d1e2-8b85-4a9e-8fd2-51f262be91a0 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:53.230258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7d1e2-8b85-4a9e-8fd2-51f262be91a0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:53.230546 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7d1e2-8b85-4a9e-8fd2-51f262be91a0 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:53.231873 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 27/51] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:25:53.236770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b25d2-932d-440c-81b9-b034acb8a771 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:53.237053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b25d2-932d-440c-81b9-b034acb8a771 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:53.237331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b25d2-932d-440c-81b9-b034acb8a771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:53.243528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:53.256745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3b8b25d2-932d-440c-81b9-b034acb8a771 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:53.257966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3b8b25d2-932d-440c-81b9-b034acb8a771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:53.296760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:53.325561 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 28/52] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:25:53.326962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3a0e7-d8f3-41e7-9583-613d2c2f2617 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:53.348097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b25d2-932d-440c-81b9-b034acb8a771 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:53.360409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ab097-4602-420b-b19f-1fc27de31332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:53.361290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ab097-4602-420b-b19f-1fc27de31332 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:53.361579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ab097-4602-420b-b19f-1fc27de31332 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:53.361854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ab097-4602-420b-b19f-1fc27de31332 None admin] PATH_INFO: `/v3/users/6bca01c7266944329ecb3cbe0887bc8e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:53.363074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ab097-4602-420b-b19f-1fc27de31332 None admin] RBAC: Authorizing `identity:get_user(user_id=6bca01c7266944329ecb3cbe0887bc8e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:53.364527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ab097-4602-420b-b19f-1fc27de31332 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:53.366211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 25/53] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:25:53 2022] GET /identity/v3/users/6bca01c7266944329ecb3cbe0887bc8e => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:53.371564 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ab097-4602-420b-b19f-1fc27de31332 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:53.445222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b25d2-932d-440c-81b9-b034acb8a771 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:53.471567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ab097-4602-420b-b19f-1fc27de31332 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:53.482326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:53.487885 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fa0fa-9e5f-4406-aff4-b6f85af61841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:53.488688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fa0fa-9e5f-4406-aff4-b6f85af61841 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:53.488970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fa0fa-9e5f-4406-aff4-b6f85af61841 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:53.489246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fa0fa-9e5f-4406-aff4-b6f85af61841 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:53.490447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fa0fa-9e5f-4406-aff4-b6f85af61841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:53.491932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fa0fa-9e5f-4406-aff4-b6f85af61841 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:53.493551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 29/54] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:25:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:25:53.499543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fa0fa-9e5f-4406-aff4-b6f85af61841 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:53.505319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9fb1b-248f-4d41-8563-24c1b104caef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:53.506143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9fb1b-248f-4d41-8563-24c1b104caef None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:53.506431 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9fb1b-248f-4d41-8563-24c1b104caef None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:53.506712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9fb1b-248f-4d41-8563-24c1b104caef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:53.507891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9fb1b-248f-4d41-8563-24c1b104caef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:53.509344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9fb1b-248f-4d41-8563-24c1b104caef None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:53.511826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 26/55] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:25:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:25:53.517541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9fb1b-248f-4d41-8563-24c1b104caef None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:53.528086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:53.599445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fa0fa-9e5f-4406-aff4-b6f85af61841 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:53.623668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9fb1b-248f-4d41-8563-24c1b104caef None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:53.633101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3445c3-17df-4c92-8dc4-b7f5115697e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:53.633770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3445c3-17df-4c92-8dc4-b7f5115697e0 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:53.634089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3445c3-17df-4c92-8dc4-b7f5115697e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:53.634373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3445c3-17df-4c92-8dc4-b7f5115697e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:53.635094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3445c3-17df-4c92-8dc4-b7f5115697e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:53.637110 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3445c3-17df-4c92-8dc4-b7f5115697e0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:53.657877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382a94e-f450-4584-adf2-d89329585442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:53.658562 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382a94e-f450-4584-adf2-d89329585442 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:53.658848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382a94e-f450-4584-adf2-d89329585442 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:53.659131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382a94e-f450-4584-adf2-d89329585442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:53.659976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382a94e-f450-4584-adf2-d89329585442 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.domain.id=default, user.id=6bca01c7266944329ecb3cbe0887bc8e) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:25:53.660294 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382a94e-f450-4584-adf2-d89329585442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:53.661730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382a94e-f450-4584-adf2-d89329585442 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:53.663647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 30/56] 10.222.0.60 () {60 vars in 1140 bytes} [Mon Jul 25 12:25:53 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:25:53.667639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 27/57] 10.222.0.60 () {56 vars in 1322 bytes} [Mon Jul 25 12:25:53 2022] GET /identity/v3/role_assignments?user.id=6bca01c7266944329ecb3cbe0887bc8e&scope.domain.id=default&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:25:55.657069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd92bf-c9c5-45c7-8868-e718db15b6a3 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:55.657568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd92bf-c9c5-45c7-8868-e718db15b6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:55.657654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd92bf-c9c5-45c7-8868-e718db15b6a3 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:55.659030 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 31/58] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:25:55.663994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c29762-b814-4c29-9358-89e3be44634e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:55.664393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c29762-b814-4c29-9358-89e3be44634e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:55.664567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c29762-b814-4c29-9358-89e3be44634e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:55.682780 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-90c29762-b814-4c29-9358-89e3be44634e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:55.684609 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-90c29762-b814-4c29-9358-89e3be44634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:55.748846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 28/59] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:25:55.771695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd92bf-c9c5-45c7-8868-e718db15b6a3 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:55.824016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:55.871683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd92bf-c9c5-45c7-8868-e718db15b6a3 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:55.903414 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b81f86-ea43-45fb-bcf8-a57eeed43f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:55.904255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b81f86-ea43-45fb-bcf8-a57eeed43f7c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:55.904535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b81f86-ea43-45fb-bcf8-a57eeed43f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:55.905097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b81f86-ea43-45fb-bcf8-a57eeed43f7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:55.906204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b81f86-ea43-45fb-bcf8-a57eeed43f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:55.907970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b81f86-ea43-45fb-bcf8-a57eeed43f7c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:55.909695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 32/60] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:25:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:55.915635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c29762-b814-4c29-9358-89e3be44634e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:55.918515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:56.014303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c29762-b814-4c29-9358-89e3be44634e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:56.047667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c58ec2-e497-49da-8a86-b8834ab3694a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:56.048334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c58ec2-e497-49da-8a86-b8834ab3694a None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:56.048624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c58ec2-e497-49da-8a86-b8834ab3694a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:56.048907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c58ec2-e497-49da-8a86-b8834ab3694a None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:56.049630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c58ec2-e497-49da-8a86-b8834ab3694a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:56.051102 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c58ec2-e497-49da-8a86-b8834ab3694a None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:56.077058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 29/61] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:25:55 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:25:59.049080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ec113-c576-4854-85b5-5a8440489e3e None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.049947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ec113-c576-4854-85b5-5a8440489e3e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.050397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ec113-c576-4854-85b5-5a8440489e3e None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.051915 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 33/62] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:25:59.057268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4326465-03f0-4d04-aa19-495a91b2c5b7 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.057664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4326465-03f0-4d04-aa19-495a91b2c5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.058599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4326465-03f0-4d04-aa19-495a91b2c5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.086941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b4326465-03f0-4d04-aa19-495a91b2c5b7 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:59.088136 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b4326465-03f0-4d04-aa19-495a91b2c5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:59.159792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:59.180234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 30/63] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:25:59.210875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ec113-c576-4854-85b5-5a8440489e3e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:59.313203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f545f-d790-4c84-b81f-11eb5f544619 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.313486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f545f-d790-4c84-b81f-11eb5f544619 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.313682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f545f-d790-4c84-b81f-11eb5f544619 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.315336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 31/64] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:25:59.327613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670833c1-d0a7-414f-a902-64983757d4da None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.327902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670833c1-d0a7-414f-a902-64983757d4da None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.328189 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670833c1-d0a7-414f-a902-64983757d4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.350262 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ec113-c576-4854-85b5-5a8440489e3e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:59.369032 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-670833c1-d0a7-414f-a902-64983757d4da None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:59.370264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-670833c1-d0a7-414f-a902-64983757d4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:59.390988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:59.419084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f08f1-1611-4dd5-b3d2-d3040c443f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:59.419890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f08f1-1611-4dd5-b3d2-d3040c443f2e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.420173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f08f1-1611-4dd5-b3d2-d3040c443f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.420449 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f08f1-1611-4dd5-b3d2-d3040c443f2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.438654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f08f1-1611-4dd5-b3d2-d3040c443f2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c16f08f1-1611-4dd5-b3d2-d3040c443f2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:25:59.446801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:25:59.447668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:25:59.448512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 34/65] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:25:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:25:59.449996 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a987a-14fe-427e-ace2-b6a455f166fd None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.450429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a987a-14fe-427e-ace2-b6a455f166fd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.450837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a987a-14fe-427e-ace2-b6a455f166fd None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.452521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 35/66] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:25:59.457699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a987a-14fe-427e-ace2-b6a455f166fd None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:59.479352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 32/67] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:25:59.481534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135dffd-d8f8-436b-b72a-5c654002b7e7 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.482068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135dffd-d8f8-436b-b72a-5c654002b7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.482565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135dffd-d8f8-436b-b72a-5c654002b7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.510898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-0135dffd-d8f8-436b-b72a-5c654002b7e7 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:59.512413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0135dffd-d8f8-436b-b72a-5c654002b7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:59.584435 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a987a-14fe-427e-ace2-b6a455f166fd None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:59.590272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 33/68] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:25:59.591351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0135dffd-d8f8-436b-b72a-5c654002b7e7 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:59.643807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:59.648715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358a944-9669-48b2-90f5-6deaeb18b31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:59.649410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358a944-9669-48b2-90f5-6deaeb18b31a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.649699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358a944-9669-48b2-90f5-6deaeb18b31a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.650444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358a944-9669-48b2-90f5-6deaeb18b31a None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.651309 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358a944-9669-48b2-90f5-6deaeb18b31a None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:25:59.651811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358a944-9669-48b2-90f5-6deaeb18b31a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:59.653339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358a944-9669-48b2-90f5-6deaeb18b31a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:59.662410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 36/69] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:25:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:59.663589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358a944-9669-48b2-90f5-6deaeb18b31a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:59.701268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0135dffd-d8f8-436b-b72a-5c654002b7e7 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:59.709240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:59.738513 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c8f90-18f8-4f22-9b6e-7876ba07aa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:59.739334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c8f90-18f8-4f22-9b6e-7876ba07aa1f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.739627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c8f90-18f8-4f22-9b6e-7876ba07aa1f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.739908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c8f90-18f8-4f22-9b6e-7876ba07aa1f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.751025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c8f90-18f8-4f22-9b6e-7876ba07aa1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-2f8c8f90-18f8-4f22-9b6e-7876ba07aa1f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:25:59.759335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:25:59.760198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:25:59.760938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 34/70] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:25:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:25:59.763245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c8f90-18f8-4f22-9b6e-7876ba07aa1f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:59.777819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358a944-9669-48b2-90f5-6deaeb18b31a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:59.811494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eac1fb-31bd-4381-b831-5551f2680677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:59.812280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eac1fb-31bd-4381-b831-5551f2680677 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.812566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eac1fb-31bd-4381-b831-5551f2680677 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.812842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eac1fb-31bd-4381-b831-5551f2680677 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.822881 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eac1fb-31bd-4381-b831-5551f2680677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a9eac1fb-31bd-4381-b831-5551f2680677 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:25:59.831073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:25:59.831936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:25:59.832691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 37/71] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:25:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:25:59.833498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a46005-9fae-4338-91cc-21d26eba1c82 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.833790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a46005-9fae-4338-91cc-21d26eba1c82 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.833996 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a46005-9fae-4338-91cc-21d26eba1c82 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.835299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 38/72] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:25:59.837673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a46005-9fae-4338-91cc-21d26eba1c82 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:59.862329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c8f90-18f8-4f22-9b6e-7876ba07aa1f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:59.871634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:59.896593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e17001-fbb4-48ed-b0bd-924d5e994789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:59.897463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e17001-fbb4-48ed-b0bd-924d5e994789 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.897748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e17001-fbb4-48ed-b0bd-924d5e994789 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.898124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e17001-fbb4-48ed-b0bd-924d5e994789 None admin] PATH_INFO: `/v3/users/9b82c4a811bc44ebab8a6b00f9536634` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.913056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e17001-fbb4-48ed-b0bd-924d5e994789 None admin] RBAC: Authorizing `identity:get_user(user_id=9b82c4a811bc44ebab8a6b00f9536634)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:59.914529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e17001-fbb4-48ed-b0bd-924d5e994789 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:59.916210 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 35/73] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:25:59 2022] GET /identity/v3/users/9b82c4a811bc44ebab8a6b00f9536634 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:59.921370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2680b1-967c-4117-b744-d8910bfb21d4 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.921370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2680b1-967c-4117-b744-d8910bfb21d4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.921370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2680b1-967c-4117-b744-d8910bfb21d4 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.921370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 36/74] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:25:59.922683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2680b1-967c-4117-b744-d8910bfb21d4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:59.942665 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a46005-9fae-4338-91cc-21d26eba1c82 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:59.944165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:59.976400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524daeb2-e98b-4eed-b641-b21a2b4ef18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:59.977064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524daeb2-e98b-4eed-b641-b21a2b4ef18a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.977352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524daeb2-e98b-4eed-b641-b21a2b4ef18a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.977626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524daeb2-e98b-4eed-b641-b21a2b4ef18a None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:59.978411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524daeb2-e98b-4eed-b641-b21a2b4ef18a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:25:59.978718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524daeb2-e98b-4eed-b641-b21a2b4ef18a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:59.980638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524daeb2-e98b-4eed-b641-b21a2b4ef18a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:59.988382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 39/75] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:25:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:59.988382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d352a4a-7ba8-4543-a9df-fd25c3795b76 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:59.990136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d352a4a-7ba8-4543-a9df-fd25c3795b76 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:59.990274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d352a4a-7ba8-4543-a9df-fd25c3795b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:00.008317 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0d352a4a-7ba8-4543-a9df-fd25c3795b76 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:00.009533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0d352a4a-7ba8-4543-a9df-fd25c3795b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:00.022756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2680b1-967c-4117-b744-d8910bfb21d4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:00.056534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337a73-5dbc-4d98-abad-0d068c61ab35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:00.057197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337a73-5dbc-4d98-abad-0d068c61ab35 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:00.057481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337a73-5dbc-4d98-abad-0d068c61ab35 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:00.058608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337a73-5dbc-4d98-abad-0d068c61ab35 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:00.059109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32337a73-5dbc-4d98-abad-0d068c61ab35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:00.059415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32337a73-5dbc-4d98-abad-0d068c61ab35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:00.060805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32337a73-5dbc-4d98-abad-0d068c61ab35 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:00.068989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 37/76] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:25:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:00.069771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337a73-5dbc-4d98-abad-0d068c61ab35 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:00.076034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 40/77] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:00.079805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc404346-affd-4ef1-9553-3f8eb44f3d4f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:00.080104 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc404346-affd-4ef1-9553-3f8eb44f3d4f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:00.080392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc404346-affd-4ef1-9553-3f8eb44f3d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:00.100357 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-dc404346-affd-4ef1-9553-3f8eb44f3d4f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:00.101477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-dc404346-affd-4ef1-9553-3f8eb44f3d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:00.104415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:00.166235 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 41/78] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:00.167962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc404346-affd-4ef1-9553-3f8eb44f3d4f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:00.186751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337a73-5dbc-4d98-abad-0d068c61ab35 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:00.196062 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:00.220814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae4324-2098-46d6-a8c5-5c70818053d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:00.221470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae4324-2098-46d6-a8c5-5c70818053d1 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:00.221755 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae4324-2098-46d6-a8c5-5c70818053d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:00.222068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae4324-2098-46d6-a8c5-5c70818053d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:00.270561 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc404346-affd-4ef1-9553-3f8eb44f3d4f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:00.288408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae4324-2098-46d6-a8c5-5c70818053d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:00.290236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae4324-2098-46d6-a8c5-5c70818053d1 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:00.303944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabde6f-9f1b-45b0-8413-f4af0e32b923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:00.304812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabde6f-9f1b-45b0-8413-f4af0e32b923 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:00.305094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabde6f-9f1b-45b0-8413-f4af0e32b923 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:00.305369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabde6f-9f1b-45b0-8413-f4af0e32b923 None admin] PATH_INFO: `/v3/users/9b82c4a811bc44ebab8a6b00f9536634` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:00.306550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabde6f-9f1b-45b0-8413-f4af0e32b923 None admin] RBAC: Authorizing `identity:get_user(user_id=9b82c4a811bc44ebab8a6b00f9536634)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:00.307991 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabde6f-9f1b-45b0-8413-f4af0e32b923 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:00.309617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 42/79] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:00 2022] GET /identity/v3/users/9b82c4a811bc44ebab8a6b00f9536634 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:00.310397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabde6f-9f1b-45b0-8413-f4af0e32b923 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:00.317279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:00.387872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 38/80] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:00.388618 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae4324-2098-46d6-a8c5-5c70818053d1 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:00.402553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:00.412008 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabde6f-9f1b-45b0-8413-f4af0e32b923 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:00.445319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a609dbe-fe39-43de-b1e5-feaf5499b1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:00.446224 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a609dbe-fe39-43de-b1e5-feaf5499b1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:00.446511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a609dbe-fe39-43de-b1e5-feaf5499b1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:00.446786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a609dbe-fe39-43de-b1e5-feaf5499b1e1 None admin] PATH_INFO: `/v3/users/6bca01c7266944329ecb3cbe0887bc8e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:00.447980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a609dbe-fe39-43de-b1e5-feaf5499b1e1 None admin] RBAC: Authorizing `identity:get_user(user_id=6bca01c7266944329ecb3cbe0887bc8e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:00.449439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a609dbe-fe39-43de-b1e5-feaf5499b1e1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:00.451098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 43/81] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:00 2022] GET /identity/v3/users/6bca01c7266944329ecb3cbe0887bc8e => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:00.451893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a609dbe-fe39-43de-b1e5-feaf5499b1e1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:00.489400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae4324-2098-46d6-a8c5-5c70818053d1 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:00.514873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:00.523062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89239752-5e76-4c52-81ee-67455ab08300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:00.523856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89239752-5e76-4c52-81ee-67455ab08300 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:00.524140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89239752-5e76-4c52-81ee-67455ab08300 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:00.524414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89239752-5e76-4c52-81ee-67455ab08300 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:00.525586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89239752-5e76-4c52-81ee-67455ab08300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:00.527598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89239752-5e76-4c52-81ee-67455ab08300 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:00.529216 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 39/82] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:00.529999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89239752-5e76-4c52-81ee-67455ab08300 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:00.552810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a609dbe-fe39-43de-b1e5-feaf5499b1e1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:00.586276 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d611b8-3d74-4f6e-bf45-80a80bc9796b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:00.587062 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d611b8-3d74-4f6e-bf45-80a80bc9796b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:00.587339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d611b8-3d74-4f6e-bf45-80a80bc9796b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:00.587818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d611b8-3d74-4f6e-bf45-80a80bc9796b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:00.597482 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d611b8-3d74-4f6e-bf45-80a80bc9796b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-48d611b8-3d74-4f6e-bf45-80a80bc9796b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:00.605445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:00.606310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:00.607068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 44/83] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:00.607679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d611b8-3d74-4f6e-bf45-80a80bc9796b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:00.629074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:00.629232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89239752-5e76-4c52-81ee-67455ab08300 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:00.662410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554db145-a0ad-44b3-9a50-e85299663d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:00.662988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554db145-a0ad-44b3-9a50-e85299663d1b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:00.663274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554db145-a0ad-44b3-9a50-e85299663d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:00.663555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554db145-a0ad-44b3-9a50-e85299663d1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:00.706660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d611b8-3d74-4f6e-bf45-80a80bc9796b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:00.722121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:00.728968 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554db145-a0ad-44b3-9a50-e85299663d1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:00.731076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554db145-a0ad-44b3-9a50-e85299663d1b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:00.741364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b0dc2-5b7f-4f24-8776-862556a3fa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:00.742243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b0dc2-5b7f-4f24-8776-862556a3fa9f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:00.742522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b0dc2-5b7f-4f24-8776-862556a3fa9f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:00.742792 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b0dc2-5b7f-4f24-8776-862556a3fa9f None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:00.743966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b0dc2-5b7f-4f24-8776-862556a3fa9f None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:00.745395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b0dc2-5b7f-4f24-8776-862556a3fa9f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:00.747832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 45/84] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:00 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:00.748420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b0dc2-5b7f-4f24-8776-862556a3fa9f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:00.832661 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:00.832913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 40/85] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:00.833735 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554db145-a0ad-44b3-9a50-e85299663d1b None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:00.844723 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b0dc2-5b7f-4f24-8776-862556a3fa9f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:00.878120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed52875-ea50-42b9-aaed-751524ddf265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:00.878762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed52875-ea50-42b9-aaed-751524ddf265 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:00.879046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed52875-ea50-42b9-aaed-751524ddf265 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:00.879321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed52875-ea50-42b9-aaed-751524ddf265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:00.928533 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:00.935668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554db145-a0ad-44b3-9a50-e85299663d1b None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:00.944686 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed52875-ea50-42b9-aaed-751524ddf265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:00.946180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed52875-ea50-42b9-aaed-751524ddf265 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:00.969376 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf9515-8fbc-4991-bfae-da88240fe095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:00.970183 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf9515-8fbc-4991-bfae-da88240fe095 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:00.970472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf9515-8fbc-4991-bfae-da88240fe095 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:00.970836 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf9515-8fbc-4991-bfae-da88240fe095 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:00.971558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf9515-8fbc-4991-bfae-da88240fe095 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:00.973012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf9515-8fbc-4991-bfae-da88240fe095 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:00.993964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 41/86] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:26:00 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:00.994952 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf9515-8fbc-4991-bfae-da88240fe095 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:01.051259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:01.066207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 46/87] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:01.067071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed52875-ea50-42b9-aaed-751524ddf265 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:01.113170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf9515-8fbc-4991-bfae-da88240fe095 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:01.158017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b9020-0510-4d82-a68c-fc8e70f1a469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:01.158676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b9020-0510-4d82-a68c-fc8e70f1a469 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:01.158974 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b9020-0510-4d82-a68c-fc8e70f1a469 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:01.159527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b9020-0510-4d82-a68c-fc8e70f1a469 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:01.160305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b9020-0510-4d82-a68c-fc8e70f1a469 None admin] RBAC: Adding query filter params (name=member) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:01.160779 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b9020-0510-4d82-a68c-fc8e70f1a469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:01.162538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b9020-0510-4d82-a68c-fc8e70f1a469 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:01.172749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 42/88] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:01.174316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b9020-0510-4d82-a68c-fc8e70f1a469 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:01.183047 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed52875-ea50-42b9-aaed-751524ddf265 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:01.192332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:01.225065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08c2c5-c841-4d26-879a-3bda8554a4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:01.225992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08c2c5-c841-4d26-879a-3bda8554a4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:01.226279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08c2c5-c841-4d26-879a-3bda8554a4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:01.226563 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08c2c5-c841-4d26-879a-3bda8554a4d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:01.227403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08c2c5-c841-4d26-879a-3bda8554a4d4 None admin] RBAC: Adding query filter params (role.id=86e07561a0854fc28db2e9f357e53d66, scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1, user.id=9b82c4a811bc44ebab8a6b00f9536634) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:01.227721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08c2c5-c841-4d26-879a-3bda8554a4d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:01.229286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08c2c5-c841-4d26-879a-3bda8554a4d4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:01.237171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 47/89] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:01 2022] GET /identity/v3/role_assignments?user.id=9b82c4a811bc44ebab8a6b00f9536634&scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1&role.id=86e07561a0854fc28db2e9f357e53d66 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:01.241943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08c2c5-c841-4d26-879a-3bda8554a4d4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:01.318690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b9020-0510-4d82-a68c-fc8e70f1a469 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:01.319365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:01.371637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b310959-4cb6-4e50-a7cf-a913b38cf516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:01.372578 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b310959-4cb6-4e50-a7cf-a913b38cf516 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:01.372875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b310959-4cb6-4e50-a7cf-a913b38cf516 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:01.373158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b310959-4cb6-4e50-a7cf-a913b38cf516 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:01.374506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b310959-4cb6-4e50-a7cf-a913b38cf516 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:01.376020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b310959-4cb6-4e50-a7cf-a913b38cf516 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:01.378046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 43/90] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:01 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:01.378780 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b310959-4cb6-4e50-a7cf-a913b38cf516 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:01.388997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08c2c5-c841-4d26-879a-3bda8554a4d4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:01.435077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abccad6-e24c-414f-a471-03f27bae4aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:01.436239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abccad6-e24c-414f-a471-03f27bae4aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:01.436529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abccad6-e24c-414f-a471-03f27bae4aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:01.436810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abccad6-e24c-414f-a471-03f27bae4aa5 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:01.438302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abccad6-e24c-414f-a471-03f27bae4aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:01.440227 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abccad6-e24c-414f-a471-03f27bae4aa5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:01.442289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 48/91] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:01 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:01.443082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abccad6-e24c-414f-a471-03f27bae4aa5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:01.484249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:01.498169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b310959-4cb6-4e50-a7cf-a913b38cf516 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:01.545443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319addf4-75f7-44c1-960e-71581007724c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:01.546819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319addf4-75f7-44c1-960e-71581007724c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:01.547163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319addf4-75f7-44c1-960e-71581007724c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:01.547446 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319addf4-75f7-44c1-960e-71581007724c None admin] PATH_INFO: `/v3/users/9b82c4a811bc44ebab8a6b00f9536634` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:01.548968 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319addf4-75f7-44c1-960e-71581007724c None admin] RBAC: Authorizing `identity:get_user(user_id=9b82c4a811bc44ebab8a6b00f9536634)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:01.551126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319addf4-75f7-44c1-960e-71581007724c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:01.553101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 44/92] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:01 2022] GET /identity/v3/users/9b82c4a811bc44ebab8a6b00f9536634 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:01.553710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319addf4-75f7-44c1-960e-71581007724c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:01.572701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abccad6-e24c-414f-a471-03f27bae4aa5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:01.595075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:01.611661 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea5ba0-914b-4f28-9664-c45118fd5dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:01.612501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea5ba0-914b-4f28-9664-c45118fd5dec None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:01.612787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea5ba0-914b-4f28-9664-c45118fd5dec None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:01.613192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea5ba0-914b-4f28-9664-c45118fd5dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:01.614059 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea5ba0-914b-4f28-9664-c45118fd5dec None admin] RBAC: Adding query filter params (role.id=950510c081de4ab6b12f383c0748450d, scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1, user.id=9b82c4a811bc44ebab8a6b00f9536634) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:01.614373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea5ba0-914b-4f28-9664-c45118fd5dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:01.617944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea5ba0-914b-4f28-9664-c45118fd5dec None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:01.625941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 49/93] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:01 2022] GET /identity/v3/role_assignments?user.id=9b82c4a811bc44ebab8a6b00f9536634&scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1&role.id=950510c081de4ab6b12f383c0748450d => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:01.625941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea5ba0-914b-4f28-9664-c45118fd5dec None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:01.742473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea5ba0-914b-4f28-9664-c45118fd5dec None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:01.807678 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c278caf-9cd6-42a5-9268-a9ce1c4f7bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:01.808886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c278caf-9cd6-42a5-9268-a9ce1c4f7bcf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:01.809176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c278caf-9cd6-42a5-9268-a9ce1c4f7bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:01.809448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c278caf-9cd6-42a5-9268-a9ce1c4f7bcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:01.867313 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319addf4-75f7-44c1-960e-71581007724c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:01.931528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:01.939243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bfd0b-cea9-4754-be5c-a1182cabea13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:01.940978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bfd0b-cea9-4754-be5c-a1182cabea13 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:01.942161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bfd0b-cea9-4754-be5c-a1182cabea13 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:01.942800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bfd0b-cea9-4754-be5c-a1182cabea13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:01.944817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bfd0b-cea9-4754-be5c-a1182cabea13 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1, user.id=6bca01c7266944329ecb3cbe0887bc8e) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:01.945528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bfd0b-cea9-4754-be5c-a1182cabea13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:01.947983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bfd0b-cea9-4754-be5c-a1182cabea13 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:01.957594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 45/94] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:01 2022] GET /identity/v3/role_assignments?user.id=6bca01c7266944329ecb3cbe0887bc8e&scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 259 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:01.968382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c278caf-9cd6-42a5-9268-a9ce1c4f7bcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:01.970332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c278caf-9cd6-42a5-9268-a9ce1c4f7bcf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:02.114613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 50/95] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:02.119870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bfd0b-cea9-4754-be5c-a1182cabea13 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:02.153962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:02.239814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bfd0b-cea9-4754-be5c-a1182cabea13 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:02.275944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fe35e-b8d6-4240-8b33-5e178be19df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:02.277214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fe35e-b8d6-4240-8b33-5e178be19df8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:02.277703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fe35e-b8d6-4240-8b33-5e178be19df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:02.278216 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fe35e-b8d6-4240-8b33-5e178be19df8 None admin] PATH_INFO: `/v3/projects/040fe7f7a2fe4f9795fdc754be42c648` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:02.279640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6fe35e-b8d6-4240-8b33-5e178be19df8 None admin] RBAC: Authorizing `identity:get_project(project_id=040fe7f7a2fe4f9795fdc754be42c648)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:02.281329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6fe35e-b8d6-4240-8b33-5e178be19df8 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:02.283290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 46/96] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:02 2022] GET /identity/v3/projects/040fe7f7a2fe4f9795fdc754be42c648 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:02.289062 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c278caf-9cd6-42a5-9268-a9ce1c4f7bcf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:02.405073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:02.423585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c278caf-9cd6-42a5-9268-a9ce1c4f7bcf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:02.495191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154962fd-9068-45f9-9a93-7b40258dd664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:02.496414 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154962fd-9068-45f9-9a93-7b40258dd664 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:02.496703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154962fd-9068-45f9-9a93-7b40258dd664 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:02.496985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154962fd-9068-45f9-9a93-7b40258dd664 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:02.497832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154962fd-9068-45f9-9a93-7b40258dd664 None admin] RBAC: Adding query filter params (role.id=86e07561a0854fc28db2e9f357e53d66, scope.project.id=040fe7f7a2fe4f9795fdc754be42c648, user.id=9b82c4a811bc44ebab8a6b00f9536634) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:02.498248 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154962fd-9068-45f9-9a93-7b40258dd664 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:02.501522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154962fd-9068-45f9-9a93-7b40258dd664 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:02.510053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 51/97] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:02 2022] GET /identity/v3/role_assignments?user.id=9b82c4a811bc44ebab8a6b00f9536634&scope.project.id=040fe7f7a2fe4f9795fdc754be42c648&role.id=86e07561a0854fc28db2e9f357e53d66 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:03.825496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b9cdb-8880-48be-80ca-28915075986c None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:03.826051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b9cdb-8880-48be-80ca-28915075986c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:03.826133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b9cdb-8880-48be-80ca-28915075986c None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:03.827594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 47/98] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:03.833064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd7736-d8bb-4365-a21d-7cdfc034c66c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:03.833482 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd7736-d8bb-4365-a21d-7cdfc034c66c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:03.833671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd7736-d8bb-4365-a21d-7cdfc034c66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:03.856857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3ccd7736-d8bb-4365-a21d-7cdfc034c66c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:03.858399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3ccd7736-d8bb-4365-a21d-7cdfc034c66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:03.927890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365ea28-7f8b-4ceb-a07b-34332e07f493 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:03.928196 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365ea28-7f8b-4ceb-a07b-34332e07f493 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:03.928536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365ea28-7f8b-4ceb-a07b-34332e07f493 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:03.929858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 48/99] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:26:03.932332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 52/100] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:03.936261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa876068-cfbb-4acc-ba89-b747f08b1366 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:03.936559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa876068-cfbb-4acc-ba89-b747f08b1366 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:03.936846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa876068-cfbb-4acc-ba89-b747f08b1366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:03.956755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd7736-d8bb-4365-a21d-7cdfc034c66c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:03.970004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-fa876068-cfbb-4acc-ba89-b747f08b1366 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:03.970004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fa876068-cfbb-4acc-ba89-b747f08b1366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:04.059683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:04.080470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 49/101] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:04.084659 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd7736-d8bb-4365-a21d-7cdfc034c66c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:04.127426 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa876068-cfbb-4acc-ba89-b747f08b1366 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:04.129938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd063c8c-0724-417e-964d-7610497d1df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:04.130869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd063c8c-0724-417e-964d-7610497d1df3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:04.131161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd063c8c-0724-417e-964d-7610497d1df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:04.131444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd063c8c-0724-417e-964d-7610497d1df3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:04.148848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:04.169424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd063c8c-0724-417e-964d-7610497d1df3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-cd063c8c-0724-417e-964d-7610497d1df3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:04.175328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:04.176162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:04.176897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:04.176897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:04.176897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:04.176897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:04.176897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:04.176897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:04.176897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:04.176897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:04.176897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:04.176897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:04.176897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:04.176897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:04.184867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 53/102] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:26:04.193216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd063c8c-0724-417e-964d-7610497d1df3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:04.314819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa876068-cfbb-4acc-ba89-b747f08b1366 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:04.391933 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd063c8c-0724-417e-964d-7610497d1df3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:04.394321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afef35-fb21-443e-8bb9-19a93585251f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:04.395133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afef35-fb21-443e-8bb9-19a93585251f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:04.395422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afef35-fb21-443e-8bb9-19a93585251f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:04.395697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afef35-fb21-443e-8bb9-19a93585251f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:04.415076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afef35-fb21-443e-8bb9-19a93585251f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-c1afef35-fb21-443e-8bb9-19a93585251f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:04.429014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:04.429868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:04.430655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 50/103] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:26:04.432023 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f96a2-3a4e-468e-90ef-b091a250aed6 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:04.432023 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f96a2-3a4e-468e-90ef-b091a250aed6 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:04.432183 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f96a2-3a4e-468e-90ef-b091a250aed6 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:04.446876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 51/104] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:04 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:04.449159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f96a2-3a4e-468e-90ef-b091a250aed6 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:04.465518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996cd9c-1e04-4234-ac46-aa7e5affa311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:04.466246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996cd9c-1e04-4234-ac46-aa7e5affa311 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:04.466538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996cd9c-1e04-4234-ac46-aa7e5affa311 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:04.466821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996cd9c-1e04-4234-ac46-aa7e5affa311 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:04.467603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996cd9c-1e04-4234-ac46-aa7e5affa311 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:04.467918 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996cd9c-1e04-4234-ac46-aa7e5affa311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:04.469336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996cd9c-1e04-4234-ac46-aa7e5affa311 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:04.482422 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:04.489369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 54/105] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:04.490932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3cc4c-a978-4d04-b30c-82de63861af6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:04.491228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3cc4c-a978-4d04-b30c-82de63861af6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:04.491518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3cc4c-a978-4d04-b30c-82de63861af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:04.527011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-29d3cc4c-a978-4d04-b30c-82de63861af6 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:04.528230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-29d3cc4c-a978-4d04-b30c-82de63861af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:04.551807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:04.587623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f96a2-3a4e-468e-90ef-b091a250aed6 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:04.624887 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c6e0e-858f-4086-ae39-a33990497dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:04.625566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c6e0e-858f-4086-ae39-a33990497dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:04.625857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c6e0e-858f-4086-ae39-a33990497dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:04.626170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c6e0e-858f-4086-ae39-a33990497dc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:04.627034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c6e0e-858f-4086-ae39-a33990497dc4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:04.627353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c6e0e-858f-4086-ae39-a33990497dc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:04.628854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c6e0e-858f-4086-ae39-a33990497dc4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:04.633178 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 55/106] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:04.634391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3cc4c-a978-4d04-b30c-82de63861af6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:04.639381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 52/107] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:04.640778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68690910-d7ee-48a8-978b-b98746d676d8 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:04.641073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68690910-d7ee-48a8-978b-b98746d676d8 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:04.641356 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68690910-d7ee-48a8-978b-b98746d676d8 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:04.642884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 53/108] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:04.647496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68690910-d7ee-48a8-978b-b98746d676d8 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:04.758270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68690910-d7ee-48a8-978b-b98746d676d8 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:04.759756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3cc4c-a978-4d04-b30c-82de63861af6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:04.793674 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab4143-257c-48be-8e26-3202f8cb116f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:04.794658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab4143-257c-48be-8e26-3202f8cb116f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:04.794887 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab4143-257c-48be-8e26-3202f8cb116f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:04.795165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab4143-257c-48be-8e26-3202f8cb116f None admin] PATH_INFO: `/v3/users/9b82c4a811bc44ebab8a6b00f9536634` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:04.796389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab4143-257c-48be-8e26-3202f8cb116f None admin] RBAC: Authorizing `identity:get_user(user_id=9b82c4a811bc44ebab8a6b00f9536634)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:04.797695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:04.797867 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab4143-257c-48be-8e26-3202f8cb116f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:04.799932 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 54/109] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:04 2022] GET /identity/v3/users/9b82c4a811bc44ebab8a6b00f9536634 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:04.801181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc608e-2bc4-4a87-b487-fa7f3779d23f None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:04.801476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc608e-2bc4-4a87-b487-fa7f3779d23f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:04.801791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc608e-2bc4-4a87-b487-fa7f3779d23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:04.804987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecf34b-fafc-46dd-9cb0-ed7a47516a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:04.806537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecf34b-fafc-46dd-9cb0-ed7a47516a20 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:04.807081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecf34b-fafc-46dd-9cb0-ed7a47516a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:04.807552 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecf34b-fafc-46dd-9cb0-ed7a47516a20 None admin] PATH_INFO: `/v3/users/bc0dcabcdc0241759d773e360b0cb0d6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:04.809876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:04.824956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b6dc608e-2bc4-4a87-b487-fa7f3779d23f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:04.826057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecf34b-fafc-46dd-9cb0-ed7a47516a20 None admin] RBAC: Authorizing `identity:get_user(user_id=bc0dcabcdc0241759d773e360b0cb0d6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:04.826246 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b6dc608e-2bc4-4a87-b487-fa7f3779d23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:04.828427 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecf34b-fafc-46dd-9cb0-ed7a47516a20 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:04.831425 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 56/110] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:04 2022] GET /identity/v3/users/bc0dcabcdc0241759d773e360b0cb0d6 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:04.836007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecf34b-fafc-46dd-9cb0-ed7a47516a20 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:04.906898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 55/111] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:04.912604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc608e-2bc4-4a87-b487-fa7f3779d23f None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:04.954570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecf34b-fafc-46dd-9cb0-ed7a47516a20 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:04.997356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8081b-9c74-4d23-907c-47c976a3e446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:04.998670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8081b-9c74-4d23-907c-47c976a3e446 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:04.999260 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8081b-9c74-4d23-907c-47c976a3e446 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:04.999679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8081b-9c74-4d23-907c-47c976a3e446 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.001568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8081b-9c74-4d23-907c-47c976a3e446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:05.003200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8081b-9c74-4d23-907c-47c976a3e446 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:05.005630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 57/112] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:05.006839 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8081b-9c74-4d23-907c-47c976a3e446 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:05.048787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc608e-2bc4-4a87-b487-fa7f3779d23f None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:05.072359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:05.087170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:05.094845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f9cb6-e7ee-4a9a-ad57-e08c13ca8e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:05.095781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f9cb6-e7ee-4a9a-ad57-e08c13ca8e1c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:05.096068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f9cb6-e7ee-4a9a-ad57-e08c13ca8e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:05.096349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f9cb6-e7ee-4a9a-ad57-e08c13ca8e1c None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.097625 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f9cb6-e7ee-4a9a-ad57-e08c13ca8e1c None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:05.099154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f9cb6-e7ee-4a9a-ad57-e08c13ca8e1c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:05.101199 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 56/113] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:04 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:05.102150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f9cb6-e7ee-4a9a-ad57-e08c13ca8e1c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:05.129693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8081b-9c74-4d23-907c-47c976a3e446 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:05.171629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7b484-51ba-4f4f-9b52-e4ec38b2fb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:05.172227 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7b484-51ba-4f4f-9b52-e4ec38b2fb62 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:05.172517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7b484-51ba-4f4f-9b52-e4ec38b2fb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:05.172801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7b484-51ba-4f4f-9b52-e4ec38b2fb62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.210694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f9cb6-e7ee-4a9a-ad57-e08c13ca8e1c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:05.248345 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aae938-548c-453d-a8aa-1289cf273632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:05.249159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aae938-548c-453d-a8aa-1289cf273632 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:05.249442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aae938-548c-453d-a8aa-1289cf273632 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:05.249717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aae938-548c-453d-a8aa-1289cf273632 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.251316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7b484-51ba-4f4f-9b52-e4ec38b2fb62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:05.252930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7b484-51ba-4f4f-9b52-e4ec38b2fb62 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:05.260361 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aae938-548c-453d-a8aa-1289cf273632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-57aae938-548c-453d-a8aa-1289cf273632 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:05.268373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:05.269213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:05.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 57/114] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:26:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:05.270615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aae938-548c-453d-a8aa-1289cf273632 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:05.314875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:05.337516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:05.368226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 58/115] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:05.368226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7b484-51ba-4f4f-9b52-e4ec38b2fb62 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:05.385451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aae938-548c-453d-a8aa-1289cf273632 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:05.419429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65afa7d-d926-4772-b2f5-e213561484de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:05.420098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65afa7d-d926-4772-b2f5-e213561484de None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:05.420395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65afa7d-d926-4772-b2f5-e213561484de None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:05.420680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65afa7d-d926-4772-b2f5-e213561484de None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.421406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65afa7d-d926-4772-b2f5-e213561484de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:05.422966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65afa7d-d926-4772-b2f5-e213561484de None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:05.452309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 58/116] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:26:05 2022] POST /identity/v3/projects => generated 302 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:05.463826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12229a13-1b23-4b97-b7f4-c75f5b611c1d None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:05.464206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12229a13-1b23-4b97-b7f4-c75f5b611c1d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:05.464556 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12229a13-1b23-4b97-b7f4-c75f5b611c1d None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.466107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 59/117] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:05.466886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12229a13-1b23-4b97-b7f4-c75f5b611c1d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:05.471136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7b484-51ba-4f4f-9b52-e4ec38b2fb62 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:05.510486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd614fb1-2843-49cd-aef8-2a3e9e74af0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:05.511477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614fb1-2843-49cd-aef8-2a3e9e74af0d None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:05.511766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614fb1-2843-49cd-aef8-2a3e9e74af0d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:05.512051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614fb1-2843-49cd-aef8-2a3e9e74af0d None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1/users/9b82c4a811bc44ebab8a6b00f9536634/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.514281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd614fb1-2843-49cd-aef8-2a3e9e74af0d None admin] RBAC: Authorizing `identity:create_grant(project_id=e080e6f49f52471c81ff5d29b80d3bb1, user_id=9b82c4a811bc44ebab8a6b00f9536634, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:05.515802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd614fb1-2843-49cd-aef8-2a3e9e74af0d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:05.541965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 59/118] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:26:05 2022] PUT /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1/users/9b82c4a811bc44ebab8a6b00f9536634/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:05.541965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd614fb1-2843-49cd-aef8-2a3e9e74af0d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:05.584474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:05.603059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12229a13-1b23-4b97-b7f4-c75f5b611c1d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:05.623688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:05.657692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82964c4d-68c2-49ca-8671-be887c36a9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:05.658440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82964c4d-68c2-49ca-8671-be887c36a9fa None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:05.658723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82964c4d-68c2-49ca-8671-be887c36a9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:05.658997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82964c4d-68c2-49ca-8671-be887c36a9fa None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.659897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82964c4d-68c2-49ca-8671-be887c36a9fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:05.660062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82964c4d-68c2-49ca-8671-be887c36a9fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:05.661542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82964c4d-68c2-49ca-8671-be887c36a9fa None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:05.678142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 60/119] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:05.680149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2ba91-ae29-417e-82d0-384d8fb8c0bb None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:05.680633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2ba91-ae29-417e-82d0-384d8fb8c0bb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:05.681248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2ba91-ae29-417e-82d0-384d8fb8c0bb None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.684151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 61/120] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:05.685610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e996e9c-52c2-44d8-b12f-76346f31c894 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:05.686111 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e996e9c-52c2-44d8-b12f-76346f31c894 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:05.686562 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e996e9c-52c2-44d8-b12f-76346f31c894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.711871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd614fb1-2843-49cd-aef8-2a3e9e74af0d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:05.716353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-6e996e9c-52c2-44d8-b12f-76346f31c894 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:05.717579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6e996e9c-52c2-44d8-b12f-76346f31c894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:05.754972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1880f-8a3f-4be2-8116-a6ef6458d9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:05.756404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1880f-8a3f-4be2-8116-a6ef6458d9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:05.757791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1880f-8a3f-4be2-8116-a6ef6458d9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:05.758281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1880f-8a3f-4be2-8116-a6ef6458d9e2 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.759720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1880f-8a3f-4be2-8116-a6ef6458d9e2 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:05.761959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1880f-8a3f-4be2-8116-a6ef6458d9e2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:05.764463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 60/121] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:05 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:05.765904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1880f-8a3f-4be2-8116-a6ef6458d9e2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:05.795624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 62/122] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:05.797107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fd197-32a6-4c9c-842c-6ebcfe970cdb None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:05.797386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fd197-32a6-4c9c-842c-6ebcfe970cdb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:05.797666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fd197-32a6-4c9c-842c-6ebcfe970cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.816890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-dc3fd197-32a6-4c9c-842c-6ebcfe970cdb None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:05.818144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-dc3fd197-32a6-4c9c-842c-6ebcfe970cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:05.866446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:05.880737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1880f-8a3f-4be2-8116-a6ef6458d9e2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:05.886707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 63/123] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:05.887558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fd197-32a6-4c9c-842c-6ebcfe970cdb None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:05.923961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861caed4-e9c3-47e7-8ef1-a3aafbc6c190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:05.925161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861caed4-e9c3-47e7-8ef1-a3aafbc6c190 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:05.925478 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861caed4-e9c3-47e7-8ef1-a3aafbc6c190 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:05.925768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861caed4-e9c3-47e7-8ef1-a3aafbc6c190 None admin] PATH_INFO: `/v3/users/9b82c4a811bc44ebab8a6b00f9536634` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:05.927068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861caed4-e9c3-47e7-8ef1-a3aafbc6c190 None admin] RBAC: Authorizing `identity:get_user(user_id=9b82c4a811bc44ebab8a6b00f9536634)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:05.927963 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:05.930417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861caed4-e9c3-47e7-8ef1-a3aafbc6c190 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:05.932135 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 61/124] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:05 2022] GET /identity/v3/users/9b82c4a811bc44ebab8a6b00f9536634 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:05.937275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861caed4-e9c3-47e7-8ef1-a3aafbc6c190 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:06.018594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fd197-32a6-4c9c-842c-6ebcfe970cdb None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:06.057312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964985a-fcc9-4427-9d1e-74ae2f67e63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:06.058548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964985a-fcc9-4427-9d1e-74ae2f67e63f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:06.058846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964985a-fcc9-4427-9d1e-74ae2f67e63f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:06.059141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964985a-fcc9-4427-9d1e-74ae2f67e63f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:06.059963 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964985a-fcc9-4427-9d1e-74ae2f67e63f None admin] RBAC: Adding query filter params (role.id=06b987ffe3ed42d1bc1963d1459eaaa6, scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1, user.id=bc0dcabcdc0241759d773e360b0cb0d6) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:06.060275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964985a-fcc9-4427-9d1e-74ae2f67e63f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:06.061725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964985a-fcc9-4427-9d1e-74ae2f67e63f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:06.067397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 64/125] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:05 2022] GET /identity/v3/role_assignments?user.id=bc0dcabcdc0241759d773e360b0cb0d6&scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1&role.id=06b987ffe3ed42d1bc1963d1459eaaa6 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:06.068179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964985a-fcc9-4427-9d1e-74ae2f67e63f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:06.073361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861caed4-e9c3-47e7-8ef1-a3aafbc6c190 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:06.124299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994b8e1-378d-43c5-895a-837bfe8cafdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:06.127324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994b8e1-378d-43c5-895a-837bfe8cafdf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:06.127613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994b8e1-378d-43c5-895a-837bfe8cafdf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:06.128603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994b8e1-378d-43c5-895a-837bfe8cafdf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:06.178614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994b8e1-378d-43c5-895a-837bfe8cafdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-0994b8e1-378d-43c5-895a-837bfe8cafdf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:06.190942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:06.192021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:06.192785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:06.192785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:06.192785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:06.192785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:06.192785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:06.192785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:06.192785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:06.192785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:06.192785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:06.192785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:06.192785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:06.192785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:06.194769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 62/126] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:26:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:06.196440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994b8e1-378d-43c5-895a-837bfe8cafdf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:06.221889 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:06.230768 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964985a-fcc9-4427-9d1e-74ae2f67e63f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:06.234554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:06.298227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02103171-c1ef-483b-bf4d-e245e967cde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:06.299314 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02103171-c1ef-483b-bf4d-e245e967cde1 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:06.299599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02103171-c1ef-483b-bf4d-e245e967cde1 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:06.299875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02103171-c1ef-483b-bf4d-e245e967cde1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:06.316327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02103171-c1ef-483b-bf4d-e245e967cde1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-02103171-c1ef-483b-bf4d-e245e967cde1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:06.327303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:06.328168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:06.328925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 65/127] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:06.329552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02103171-c1ef-483b-bf4d-e245e967cde1 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:06.388040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994b8e1-378d-43c5-895a-837bfe8cafdf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:06.425485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db64f2-acb4-42b5-b4ba-a8c32727c363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:06.427018 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db64f2-acb4-42b5-b4ba-a8c32727c363 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:06.427562 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db64f2-acb4-42b5-b4ba-a8c32727c363 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:06.428045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db64f2-acb4-42b5-b4ba-a8c32727c363 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:06.429651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db64f2-acb4-42b5-b4ba-a8c32727c363 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:06.431480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db64f2-acb4-42b5-b4ba-a8c32727c363 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:06.433995 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 63/128] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:06 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:06.435509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db64f2-acb4-42b5-b4ba-a8c32727c363 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:06.530823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:06.551525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db64f2-acb4-42b5-b4ba-a8c32727c363 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:06.592927 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02103171-c1ef-483b-bf4d-e245e967cde1 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:06.604437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c927e-57f0-4993-8024-d711aa744c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:06.605440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c927e-57f0-4993-8024-d711aa744c9d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:06.605728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c927e-57f0-4993-8024-d711aa744c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:06.606033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c927e-57f0-4993-8024-d711aa744c9d None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:06.606793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c927e-57f0-4993-8024-d711aa744c9d None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:06.607102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c927e-57f0-4993-8024-d711aa744c9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:06.608497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c927e-57f0-4993-8024-d711aa744c9d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:06.624395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 64/129] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:06.627995 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c927e-57f0-4993-8024-d711aa744c9d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:06.655278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b45b1-db26-4c75-9434-cd82cff45483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:06.655963 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b45b1-db26-4c75-9434-cd82cff45483 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:06.656247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b45b1-db26-4c75-9434-cd82cff45483 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:06.656529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b45b1-db26-4c75-9434-cd82cff45483 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:06.657290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b45b1-db26-4c75-9434-cd82cff45483 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:06.657603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b45b1-db26-4c75-9434-cd82cff45483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:06.659034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b45b1-db26-4c75-9434-cd82cff45483 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:06.676831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 66/130] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:06.677755 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b45b1-db26-4c75-9434-cd82cff45483 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:06.703058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:06.764106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c927e-57f0-4993-8024-d711aa744c9d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:06.789781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b45b1-db26-4c75-9434-cd82cff45483 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:06.799374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5fdcf-c00b-4455-8448-c446b54ca559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:06.800540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5fdcf-c00b-4455-8448-c446b54ca559 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:06.801016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5fdcf-c00b-4455-8448-c446b54ca559 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:06.801485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5fdcf-c00b-4455-8448-c446b54ca559 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1/users/9b82c4a811bc44ebab8a6b00f9536634/roles/950510c081de4ab6b12f383c0748450d` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:06.805511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5fdcf-c00b-4455-8448-c446b54ca559 None admin] RBAC: Authorizing `identity:create_grant(project_id=e080e6f49f52471c81ff5d29b80d3bb1, user_id=9b82c4a811bc44ebab8a6b00f9536634, role_id=950510c081de4ab6b12f383c0748450d)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:06.808082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5fdcf-c00b-4455-8448-c446b54ca559 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:06.830052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 65/131] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:26:06 2022] PUT /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1/users/9b82c4a811bc44ebab8a6b00f9536634/roles/950510c081de4ab6b12f383c0748450d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:06.831416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5fdcf-c00b-4455-8448-c446b54ca559 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:06.835089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:06.846443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f6177-78ee-4e04-9bd3-9f87e20cadf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:06.847781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f6177-78ee-4e04-9bd3-9f87e20cadf6 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:06.847781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f6177-78ee-4e04-9bd3-9f87e20cadf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:06.848042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f6177-78ee-4e04-9bd3-9f87e20cadf6 None admin] PATH_INFO: `/v3/users/9b82c4a811bc44ebab8a6b00f9536634` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:06.849273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f6177-78ee-4e04-9bd3-9f87e20cadf6 None admin] RBAC: Authorizing `identity:get_user(user_id=9b82c4a811bc44ebab8a6b00f9536634)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:06.856840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f6177-78ee-4e04-9bd3-9f87e20cadf6 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:06.858807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 67/132] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:06 2022] GET /identity/v3/users/9b82c4a811bc44ebab8a6b00f9536634 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:06.864440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f6177-78ee-4e04-9bd3-9f87e20cadf6 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:06.959659 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5fdcf-c00b-4455-8448-c446b54ca559 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:07.001469 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5d47f-e5d8-4ea0-83e6-ad139a52a69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:07.003374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5d47f-e5d8-4ea0-83e6-ad139a52a69d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:07.003924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5d47f-e5d8-4ea0-83e6-ad139a52a69d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:07.004611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5d47f-e5d8-4ea0-83e6-ad139a52a69d None admin] PATH_INFO: `/v3/users/6bca01c7266944329ecb3cbe0887bc8e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:07.006233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5d47f-e5d8-4ea0-83e6-ad139a52a69d None admin] RBAC: Authorizing `identity:get_user(user_id=6bca01c7266944329ecb3cbe0887bc8e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:07.008099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5d47f-e5d8-4ea0-83e6-ad139a52a69d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:07.010378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 66/133] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:06 2022] GET /identity/v3/users/6bca01c7266944329ecb3cbe0887bc8e => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:07.015650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5d47f-e5d8-4ea0-83e6-ad139a52a69d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:07.109423 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:07.135567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f6177-78ee-4e04-9bd3-9f87e20cadf6 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:07.170272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:07.213318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e2813-654c-4a22-960b-12979b9fe105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:07.215288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e2813-654c-4a22-960b-12979b9fe105 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:07.215852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e2813-654c-4a22-960b-12979b9fe105 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:07.216341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e2813-654c-4a22-960b-12979b9fe105 None admin] PATH_INFO: `/v3/projects/040fe7f7a2fe4f9795fdc754be42c648` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:07.218393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e2813-654c-4a22-960b-12979b9fe105 None admin] RBAC: Authorizing `identity:get_project(project_id=040fe7f7a2fe4f9795fdc754be42c648)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:07.220710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e2813-654c-4a22-960b-12979b9fe105 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:07.223686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 68/134] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:06 2022] GET /identity/v3/projects/040fe7f7a2fe4f9795fdc754be42c648 => generated 312 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:07.230782 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e2813-654c-4a22-960b-12979b9fe105 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:07.238417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5d47f-e5d8-4ea0-83e6-ad139a52a69d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:07.315501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad53003-dae6-45a4-96f2-7e8693eb6ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:07.316382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad53003-dae6-45a4-96f2-7e8693eb6ebf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:07.316742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad53003-dae6-45a4-96f2-7e8693eb6ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:07.317015 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad53003-dae6-45a4-96f2-7e8693eb6ebf None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:07.318349 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad53003-dae6-45a4-96f2-7e8693eb6ebf None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:07.319813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad53003-dae6-45a4-96f2-7e8693eb6ebf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:07.323319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 67/135] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:07 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:07.329434 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad53003-dae6-45a4-96f2-7e8693eb6ebf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:07.412775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e2813-654c-4a22-960b-12979b9fe105 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:07.475419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c91301-2b7b-4d76-ac70-e52303aaa57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:07.476345 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c91301-2b7b-4d76-ac70-e52303aaa57c None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:07.476634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c91301-2b7b-4d76-ac70-e52303aaa57c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:07.476917 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c91301-2b7b-4d76-ac70-e52303aaa57c None admin] PATH_INFO: `/v3/projects/040fe7f7a2fe4f9795fdc754be42c648/users/9b82c4a811bc44ebab8a6b00f9536634/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:07.483448 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c91301-2b7b-4d76-ac70-e52303aaa57c None admin] RBAC: Authorizing `identity:create_grant(project_id=040fe7f7a2fe4f9795fdc754be42c648, user_id=9b82c4a811bc44ebab8a6b00f9536634, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:07.485368 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c91301-2b7b-4d76-ac70-e52303aaa57c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:07.508794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 69/136] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:26:07 2022] PUT /identity/v3/projects/040fe7f7a2fe4f9795fdc754be42c648/users/9b82c4a811bc44ebab8a6b00f9536634/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:26:07.588267 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad53003-dae6-45a4-96f2-7e8693eb6ebf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:07.643897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf304fc-4c8d-49ee-8f5e-2c6e607c49dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:07.644809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf304fc-4c8d-49ee-8f5e-2c6e607c49dc None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:07.645096 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf304fc-4c8d-49ee-8f5e-2c6e607c49dc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:07.645374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf304fc-4c8d-49ee-8f5e-2c6e607c49dc None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1/users/6bca01c7266944329ecb3cbe0887bc8e/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:07.647389 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf304fc-4c8d-49ee-8f5e-2c6e607c49dc None admin] RBAC: Authorizing `identity:create_grant(project_id=e080e6f49f52471c81ff5d29b80d3bb1, user_id=6bca01c7266944329ecb3cbe0887bc8e, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:07.648866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf304fc-4c8d-49ee-8f5e-2c6e607c49dc None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:07.669752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:07.675910 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 68/137] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:26:07 2022] PUT /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1/users/6bca01c7266944329ecb3cbe0887bc8e/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:08.175324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7658f71-bb38-457d-8a80-f29789d629b6 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:08.176346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7658f71-bb38-457d-8a80-f29789d629b6 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:08.176853 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7658f71-bb38-457d-8a80-f29789d629b6 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:08.178528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 70/138] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:08.188530 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c062f-40ff-4b4e-8efc-26c920ab13c9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:08.189341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c062f-40ff-4b4e-8efc-26c920ab13c9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:08.189841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c062f-40ff-4b4e-8efc-26c920ab13c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:08.195395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee513589-2e40-4f34-b478-0916ec3bbd7c None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:08.196285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee513589-2e40-4f34-b478-0916ec3bbd7c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:08.197298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee513589-2e40-4f34-b478-0916ec3bbd7c None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:08.199427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 71/139] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:08.206270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859f113-e71c-4637-8a21-9d15b2074b9a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:08.207910 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859f113-e71c-4637-8a21-9d15b2074b9a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:08.214473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859f113-e71c-4637-8a21-9d15b2074b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:08.238328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-122c062f-40ff-4b4e-8efc-26c920ab13c9 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:08.239822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-122c062f-40ff-4b4e-8efc-26c920ab13c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:08.253113 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-c859f113-e71c-4637-8a21-9d15b2074b9a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:08.254580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c859f113-e71c-4637-8a21-9d15b2074b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:08.278278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:08.370140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 69/140] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:08.396451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 72/141] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:08.405448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c062f-40ff-4b4e-8efc-26c920ab13c9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:08.443771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859f113-e71c-4637-8a21-9d15b2074b9a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:08.520672 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c062f-40ff-4b4e-8efc-26c920ab13c9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:08.562413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:08.578553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6fde9-5362-4601-8f58-8b61505b820f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:08.579367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6fde9-5362-4601-8f58-8b61505b820f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:08.579652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6fde9-5362-4601-8f58-8b61505b820f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:08.579930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6fde9-5362-4601-8f58-8b61505b820f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:08.581901 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6fde9-5362-4601-8f58-8b61505b820f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:08.583397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6fde9-5362-4601-8f58-8b61505b820f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:08.585273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 70/142] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:08.590975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6fde9-5362-4601-8f58-8b61505b820f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:08.633566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859f113-e71c-4637-8a21-9d15b2074b9a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:08.698834 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebac637-04f0-47aa-a23d-6ae3adfbae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:08.699969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebac637-04f0-47aa-a23d-6ae3adfbae4b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:08.701215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebac637-04f0-47aa-a23d-6ae3adfbae4b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:08.704409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebac637-04f0-47aa-a23d-6ae3adfbae4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:08.723083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebac637-04f0-47aa-a23d-6ae3adfbae4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-cebac637-04f0-47aa-a23d-6ae3adfbae4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:08.736033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:08.736865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:08.737626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:08.737626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:08.737626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:08.737626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:08.737626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:08.737626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:08.737626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:08.737626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:08.737626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:08.737626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:08.737626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:08.737626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:08.742229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 73/143] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:08.753155 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541a0a6-199b-4668-a5ea-c6c92d3c1cbc None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:08.753155 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541a0a6-199b-4668-a5ea-c6c92d3c1cbc None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:08.753155 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541a0a6-199b-4668-a5ea-c6c92d3c1cbc None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:08.753155 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 74/144] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:08.757722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541a0a6-199b-4668-a5ea-c6c92d3c1cbc None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:08.773687 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:08.805898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6fde9-5362-4601-8f58-8b61505b820f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:08.892600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541a0a6-199b-4668-a5ea-c6c92d3c1cbc None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:08.894624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84271e-b2c6-46c8-9909-aca19649f902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:08.895328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84271e-b2c6-46c8-9909-aca19649f902 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:08.896269 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84271e-b2c6-46c8-9909-aca19649f902 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:08.896269 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84271e-b2c6-46c8-9909-aca19649f902 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:08.896625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84271e-b2c6-46c8-9909-aca19649f902 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:08.898114 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84271e-b2c6-46c8-9909-aca19649f902 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:08.928760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948770cd-adcd-4042-b658-e25cbd002c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:08.929772 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948770cd-adcd-4042-b658-e25cbd002c05 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:08.930277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948770cd-adcd-4042-b658-e25cbd002c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:08.930751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948770cd-adcd-4042-b658-e25cbd002c05 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:08.931713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948770cd-adcd-4042-b658-e25cbd002c05 None admin] RBAC: Adding query filter params (name=member) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:08.932219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948770cd-adcd-4042-b658-e25cbd002c05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:08.933871 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948770cd-adcd-4042-b658-e25cbd002c05 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:08.943468 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 75/145] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:08.949120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 71/146] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:26:08 2022] POST /identity/v3/users => generated 283 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:08.950557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36730d5b-e81d-40d0-bd0b-bf1bf0275876 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:08.951050 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36730d5b-e81d-40d0-bd0b-bf1bf0275876 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:08.951522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36730d5b-e81d-40d0-bd0b-bf1bf0275876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:08.963055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84271e-b2c6-46c8-9909-aca19649f902 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:08.971763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-36730d5b-e81d-40d0-bd0b-bf1bf0275876 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:08.973314 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-36730d5b-e81d-40d0-bd0b-bf1bf0275876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:09.044370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:09.050366 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 76/147] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:09.070553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36730d5b-e81d-40d0-bd0b-bf1bf0275876 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:09.074423 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:09.170229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84271e-b2c6-46c8-9909-aca19649f902 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:09.178137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36730d5b-e81d-40d0-bd0b-bf1bf0275876 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:09.239031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92511809-1092-4309-a32d-b180ee0716b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:09.239766 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92511809-1092-4309-a32d-b180ee0716b5 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:09.240057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6169c-99a7-478b-8e25-0e9e5bc58329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:09.240503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92511809-1092-4309-a32d-b180ee0716b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:09.240503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92511809-1092-4309-a32d-b180ee0716b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:09.241662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6169c-99a7-478b-8e25-0e9e5bc58329 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:09.242288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6169c-99a7-478b-8e25-0e9e5bc58329 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:09.242774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6169c-99a7-478b-8e25-0e9e5bc58329 None admin] PATH_INFO: `/v3/users/9b82c4a811bc44ebab8a6b00f9536634` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:09.244265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6169c-99a7-478b-8e25-0e9e5bc58329 None admin] RBAC: Authorizing `identity:get_user(user_id=9b82c4a811bc44ebab8a6b00f9536634)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:09.246524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6169c-99a7-478b-8e25-0e9e5bc58329 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:09.248862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 72/148] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:08 2022] GET /identity/v3/users/9b82c4a811bc44ebab8a6b00f9536634 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:09.259638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92511809-1092-4309-a32d-b180ee0716b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:09.260886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6169c-99a7-478b-8e25-0e9e5bc58329 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-92511809-1092-4309-a32d-b180ee0716b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:09.268251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:09.269170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:09.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:09.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:09.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:09.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:09.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:09.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:09.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:09.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:09.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:09.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:09.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:09.269982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:09.274043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 77/149] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:09.281935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92511809-1092-4309-a32d-b180ee0716b5 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:09.363358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:09.441169 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6169c-99a7-478b-8e25-0e9e5bc58329 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:09.457086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92511809-1092-4309-a32d-b180ee0716b5 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:09.484114 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:09.512378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1cedc-2a6e-4978-850a-ce3f8643e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:09.512968 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1cedc-2a6e-4978-850a-ce3f8643e45c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:09.513253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1cedc-2a6e-4978-850a-ce3f8643e45c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:09.513538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1cedc-2a6e-4978-850a-ce3f8643e45c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:09.530448 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b55e6-1380-4a3d-85e9-c71f0d64692d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:09.531124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b55e6-1380-4a3d-85e9-c71f0d64692d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:09.531408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b55e6-1380-4a3d-85e9-c71f0d64692d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:09.531683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b55e6-1380-4a3d-85e9-c71f0d64692d None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:09.532521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b55e6-1380-4a3d-85e9-c71f0d64692d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:09.532835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b55e6-1380-4a3d-85e9-c71f0d64692d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:09.535784 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b55e6-1380-4a3d-85e9-c71f0d64692d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:09.550584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 78/150] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:09.560001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b55e6-1380-4a3d-85e9-c71f0d64692d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:09.649036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1cedc-2a6e-4978-850a-ce3f8643e45c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:09.651426 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1cedc-2a6e-4978-850a-ce3f8643e45c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:09.766936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b55e6-1380-4a3d-85e9-c71f0d64692d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:09.784746 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 73/151] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:09.786483 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cf06f-9220-4a69-baf1-9d7d215cc1ef None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:09.786878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cf06f-9220-4a69-baf1-9d7d215cc1ef None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:09.787163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cf06f-9220-4a69-baf1-9d7d215cc1ef None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:09.788598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 74/152] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:26:09.790998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:09.793551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cf06f-9220-4a69-baf1-9d7d215cc1ef None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:09.811754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea216f32-a4e4-42f6-9311-a4ed45557439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:09.812637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea216f32-a4e4-42f6-9311-a4ed45557439 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:09.812921 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea216f32-a4e4-42f6-9311-a4ed45557439 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:09.813198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea216f32-a4e4-42f6-9311-a4ed45557439 None admin] PATH_INFO: `/v3/users/bc0dcabcdc0241759d773e360b0cb0d6` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:09.815836 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea216f32-a4e4-42f6-9311-a4ed45557439 None admin] RBAC: Authorizing `identity:get_user(user_id=bc0dcabcdc0241759d773e360b0cb0d6)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:09.817319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea216f32-a4e4-42f6-9311-a4ed45557439 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:09.819029 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 79/153] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:09 2022] GET /identity/v3/users/bc0dcabcdc0241759d773e360b0cb0d6 => generated 289 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:09.820244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e0ece-eb8d-4179-ac53-54daf40d1623 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:09.820512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e0ece-eb8d-4179-ac53-54daf40d1623 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:09.820791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e0ece-eb8d-4179-ac53-54daf40d1623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:09.832884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:09.851700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-3d7e0ece-eb8d-4179-ac53-54daf40d1623 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:09.852908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-3d7e0ece-eb8d-4179-ac53-54daf40d1623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:09.938197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 80/154] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:09.938967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e0ece-eb8d-4179-ac53-54daf40d1623 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:09.991245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cf06f-9220-4a69-baf1-9d7d215cc1ef None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:10.054662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1f817-8c30-4fda-a313-0426e6003287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:10.055554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1f817-8c30-4fda-a313-0426e6003287 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:10.055841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1f817-8c30-4fda-a313-0426e6003287 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:10.056119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1f817-8c30-4fda-a313-0426e6003287 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:10.057389 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1f817-8c30-4fda-a313-0426e6003287 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:10.059420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1f817-8c30-4fda-a313-0426e6003287 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:10.061155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 75/155] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:09 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:10.061563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e0ece-eb8d-4179-ac53-54daf40d1623 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:10.061935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1f817-8c30-4fda-a313-0426e6003287 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:10.098938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:10.126643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832200e-521c-4c30-bad1-bf770065d0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:10.127530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832200e-521c-4c30-bad1-bf770065d0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:10.127808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832200e-521c-4c30-bad1-bf770065d0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:10.128084 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832200e-521c-4c30-bad1-bf770065d0d4 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:10.129341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832200e-521c-4c30-bad1-bf770065d0d4 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:10.130830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832200e-521c-4c30-bad1-bf770065d0d4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:10.132775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 81/156] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:09 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:10.133348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832200e-521c-4c30-bad1-bf770065d0d4 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:10.173903 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:10.187416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1f817-8c30-4fda-a313-0426e6003287 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:10.226395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3ef3d-fb5e-4959-bfce-55a0c846ca30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:10.227380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3ef3d-fb5e-4959-bfce-55a0c846ca30 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:10.227731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3ef3d-fb5e-4959-bfce-55a0c846ca30 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:10.228092 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3ef3d-fb5e-4959-bfce-55a0c846ca30 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:10.240641 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3ef3d-fb5e-4959-bfce-55a0c846ca30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-36f3ef3d-fb5e-4959-bfce-55a0c846ca30 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:10.252538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:10.253565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:10.254523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 76/157] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:26:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:10.255154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3ef3d-fb5e-4959-bfce-55a0c846ca30 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:10.340981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832200e-521c-4c30-bad1-bf770065d0d4 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:10.360595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3ef3d-fb5e-4959-bfce-55a0c846ca30 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:10.389176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69032c27-97fb-4671-a426-9c6245e659b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:10.390241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69032c27-97fb-4671-a426-9c6245e659b1 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:10.390829 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69032c27-97fb-4671-a426-9c6245e659b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:10.391321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69032c27-97fb-4671-a426-9c6245e659b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:10.392397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69032c27-97fb-4671-a426-9c6245e659b1 None admin] RBAC: Adding query filter params (role.id=86e07561a0854fc28db2e9f357e53d66, scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1, user.id=9b82c4a811bc44ebab8a6b00f9536634) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:10.392920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69032c27-97fb-4671-a426-9c6245e659b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:10.394765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69032c27-97fb-4671-a426-9c6245e659b1 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:10.404043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 82/158] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:10 2022] GET /identity/v3/role_assignments?user.id=9b82c4a811bc44ebab8a6b00f9536634&scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1&role.id=86e07561a0854fc28db2e9f357e53d66 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:10.405238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d9cf1-ac08-42a9-b948-fe3babb996e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:10.406261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d9cf1-ac08-42a9-b948-fe3babb996e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:10.406538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d9cf1-ac08-42a9-b948-fe3babb996e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:10.406805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d9cf1-ac08-42a9-b948-fe3babb996e5 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1/users/bc0dcabcdc0241759d773e360b0cb0d6/roles/06b987ffe3ed42d1bc1963d1459eaaa6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:10.407769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69032c27-97fb-4671-a426-9c6245e659b1 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:10.409708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d9cf1-ac08-42a9-b948-fe3babb996e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e080e6f49f52471c81ff5d29b80d3bb1, user_id=bc0dcabcdc0241759d773e360b0cb0d6, role_id=06b987ffe3ed42d1bc1963d1459eaaa6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:10.412726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d9cf1-ac08-42a9-b948-fe3babb996e5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:10.437857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 77/159] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:26:10 2022] PUT /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1/users/bc0dcabcdc0241759d773e360b0cb0d6/roles/06b987ffe3ed42d1bc1963d1459eaaa6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:10.487845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:10.546681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69032c27-97fb-4671-a426-9c6245e659b1 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:10.593730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4ba64-91f2-4388-a49c-defe42e4b510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:10.594439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4ba64-91f2-4388-a49c-defe42e4b510 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:10.594716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4ba64-91f2-4388-a49c-defe42e4b510 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:10.594988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4ba64-91f2-4388-a49c-defe42e4b510 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:10.595808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4ba64-91f2-4388-a49c-defe42e4b510 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:10.596113 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4ba64-91f2-4388-a49c-defe42e4b510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:10.597594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4ba64-91f2-4388-a49c-defe42e4b510 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:10.611141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 83/160] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:10.617059 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d9cf1-ac08-42a9-b948-fe3babb996e5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:10.629878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:10.736977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d9cf1-ac08-42a9-b948-fe3babb996e5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:10.779300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac7f89-d8c6-423b-9636-7d284c10c893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:10.780625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac7f89-d8c6-423b-9636-7d284c10c893 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:10.781250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac7f89-d8c6-423b-9636-7d284c10c893 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:10.781729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac7f89-d8c6-423b-9636-7d284c10c893 None admin] PATH_INFO: `/v3/users/9b82c4a811bc44ebab8a6b00f9536634` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:10.783315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac7f89-d8c6-423b-9636-7d284c10c893 None admin] RBAC: Authorizing `identity:get_user(user_id=9b82c4a811bc44ebab8a6b00f9536634)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:10.785000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac7f89-d8c6-423b-9636-7d284c10c893 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:10.786895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 78/161] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:10 2022] GET /identity/v3/users/9b82c4a811bc44ebab8a6b00f9536634 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:10.792838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4ba64-91f2-4388-a49c-defe42e4b510 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:10.839155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efba00f-c2d9-4c5a-81b8-a8ebb83e0032 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:10.839801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efba00f-c2d9-4c5a-81b8-a8ebb83e0032 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:10.840268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efba00f-c2d9-4c5a-81b8-a8ebb83e0032 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:10.842161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 79/162] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:26:10.848410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c708f4-fbfa-4d4b-be17-2b3ff08c1e93 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:10.848954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c708f4-fbfa-4d4b-be17-2b3ff08c1e93 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:10.849455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c708f4-fbfa-4d4b-be17-2b3ff08c1e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:10.882797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-24c708f4-fbfa-4d4b-be17-2b3ff08c1e93 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:10.884316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-24c708f4-fbfa-4d4b-be17-2b3ff08c1e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:10.996728 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4ba64-91f2-4388-a49c-defe42e4b510 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:11.021085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 80/163] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:11.022451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d9444-ae30-4f74-9421-5a8530c1bb7a None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:11.022735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d9444-ae30-4f74-9421-5a8530c1bb7a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:11.023019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d9444-ae30-4f74-9421-5a8530c1bb7a None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:11.024336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 81/164] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:11.038295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:11.043446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a418-b3fa-4310-b4a4-5f0ada010d90 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:11.043731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a418-b3fa-4310-b4a4-5f0ada010d90 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:11.044003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a418-b3fa-4310-b4a4-5f0ada010d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:11.068096 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:11.081105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14115177-b0c3-40b3-ae8b-2da7ed66458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:11.081762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14115177-b0c3-40b3-ae8b-2da7ed66458a None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:11.082071 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14115177-b0c3-40b3-ae8b-2da7ed66458a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:11.082347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14115177-b0c3-40b3-ae8b-2da7ed66458a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:11.082956 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-48b2a418-b3fa-4310-b4a4-5f0ada010d90 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:11.084163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-48b2a418-b3fa-4310-b4a4-5f0ada010d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:11.178943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 82/165] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:11.179740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a418-b3fa-4310-b4a4-5f0ada010d90 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:11.238985 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14115177-b0c3-40b3-ae8b-2da7ed66458a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:11.240470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14115177-b0c3-40b3-ae8b-2da7ed66458a None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:11.342554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 84/166] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:11.343410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14115177-b0c3-40b3-ae8b-2da7ed66458a None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:11.374274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:11.403096 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a418-b3fa-4310-b4a4-5f0ada010d90 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:11.444507 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fae0f6-004b-42ad-9a4a-eb2f07d4b8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:11.445614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fae0f6-004b-42ad-9a4a-eb2f07d4b8ae None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:11.446080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fae0f6-004b-42ad-9a4a-eb2f07d4b8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:11.446546 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fae0f6-004b-42ad-9a4a-eb2f07d4b8ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:11.457951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:11.475206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fae0f6-004b-42ad-9a4a-eb2f07d4b8ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:11.485937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14115177-b0c3-40b3-ae8b-2da7ed66458a None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-78fae0f6-004b-42ad-9a4a-eb2f07d4b8ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:11.488837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:11.489676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:11.490725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:11.490725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:11.490725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:11.490725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:11.490725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:11.490725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:11.490725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:11.490725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:11.490725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:11.490725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:11.490725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:11.490725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:11.493134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 83/167] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:11.494205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fae0f6-004b-42ad-9a4a-eb2f07d4b8ae None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:11.535691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38649a-3cff-4f91-90ef-5b410507c3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:11.536581 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38649a-3cff-4f91-90ef-5b410507c3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:11.536932 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38649a-3cff-4f91-90ef-5b410507c3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:11.537277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38649a-3cff-4f91-90ef-5b410507c3c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:11.554406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38649a-3cff-4f91-90ef-5b410507c3c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-4f38649a-3cff-4f91-90ef-5b410507c3c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:11.570826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:11.571719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:11.572696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 85/168] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:26:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:11.573427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38649a-3cff-4f91-90ef-5b410507c3c3 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:11.681609 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:11.724214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fae0f6-004b-42ad-9a4a-eb2f07d4b8ae None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:11.742360 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38649a-3cff-4f91-90ef-5b410507c3c3 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:11.802680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d3a35-208a-44db-9a9c-833e4ea3acd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:11.803363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d3a35-208a-44db-9a9c-833e4ea3acd9 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:11.803647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d3a35-208a-44db-9a9c-833e4ea3acd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:11.803924 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d3a35-208a-44db-9a9c-833e4ea3acd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:11.804680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d3a35-208a-44db-9a9c-833e4ea3acd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:11.804985 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d3a35-208a-44db-9a9c-833e4ea3acd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:11.806564 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d3a35-208a-44db-9a9c-833e4ea3acd9 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:11.822752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf2164-dc76-4ebd-8940-b8b0c8c7a640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:11.823324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 86/169] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:11.824724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d3a35-208a-44db-9a9c-833e4ea3acd9 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:11.824971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf2164-dc76-4ebd-8940-b8b0c8c7a640 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:11.825508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf2164-dc76-4ebd-8940-b8b0c8c7a640 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:11.826043 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf2164-dc76-4ebd-8940-b8b0c8c7a640 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:11.830704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf2164-dc76-4ebd-8940-b8b0c8c7a640 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:11.833374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf2164-dc76-4ebd-8940-b8b0c8c7a640 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:11.836977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 84/170] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:11 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:11.841935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf2164-dc76-4ebd-8940-b8b0c8c7a640 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:11.931021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d3a35-208a-44db-9a9c-833e4ea3acd9 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:11.953443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:11.976060 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17050465-2470-4552-bcc7-9f267da7e8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:11.976799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17050465-2470-4552-bcc7-9f267da7e8ec None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:11.977198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17050465-2470-4552-bcc7-9f267da7e8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:11.977570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17050465-2470-4552-bcc7-9f267da7e8ec None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:11.978439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17050465-2470-4552-bcc7-9f267da7e8ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:11.978841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17050465-2470-4552-bcc7-9f267da7e8ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:11.981092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17050465-2470-4552-bcc7-9f267da7e8ec None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:11.982506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:11.990342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 87/171] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:11.991252 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17050465-2470-4552-bcc7-9f267da7e8ec None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:11.998339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf2164-dc76-4ebd-8940-b8b0c8c7a640 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:12.034428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4a8d5-0912-4587-a485-af3b622f2fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:12.035307 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4a8d5-0912-4587-a485-af3b622f2fbc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:12.035592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4a8d5-0912-4587-a485-af3b622f2fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:12.035870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4a8d5-0912-4587-a485-af3b622f2fbc None admin] PATH_INFO: `/v3/users/9b82c4a811bc44ebab8a6b00f9536634` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:12.037188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4a8d5-0912-4587-a485-af3b622f2fbc None admin] RBAC: Authorizing `identity:get_user(user_id=9b82c4a811bc44ebab8a6b00f9536634)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:12.038810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4a8d5-0912-4587-a485-af3b622f2fbc None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:12.040577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 85/172] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:11 2022] GET /identity/v3/users/9b82c4a811bc44ebab8a6b00f9536634 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:12.041334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4a8d5-0912-4587-a485-af3b622f2fbc None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:12.108253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17050465-2470-4552-bcc7-9f267da7e8ec None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:12.142503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757b663-4722-455a-a3dd-90d2131dc18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:12.143314 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757b663-4722-455a-a3dd-90d2131dc18d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:12.143694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757b663-4722-455a-a3dd-90d2131dc18d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:12.144065 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757b663-4722-455a-a3dd-90d2131dc18d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:12.144992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757b663-4722-455a-a3dd-90d2131dc18d None admin] RBAC: Adding query filter params (role.id=950510c081de4ab6b12f383c0748450d, scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1, user.id=9b82c4a811bc44ebab8a6b00f9536634) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:12.145408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757b663-4722-455a-a3dd-90d2131dc18d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:12.147090 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757b663-4722-455a-a3dd-90d2131dc18d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:12.154279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4a8d5-0912-4587-a485-af3b622f2fbc None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:12.154644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 88/173] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:11 2022] GET /identity/v3/role_assignments?user.id=9b82c4a811bc44ebab8a6b00f9536634&scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1&role.id=950510c081de4ab6b12f383c0748450d => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:12.155562 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757b663-4722-455a-a3dd-90d2131dc18d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:12.190087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:12.197062 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111983c-5840-4183-9074-0b83085047b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:12.197968 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111983c-5840-4183-9074-0b83085047b6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:12.198258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111983c-5840-4183-9074-0b83085047b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:12.199090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111983c-5840-4183-9074-0b83085047b6 None admin] PATH_INFO: `/v3/users/6bca01c7266944329ecb3cbe0887bc8e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:12.200387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111983c-5840-4183-9074-0b83085047b6 None admin] RBAC: Authorizing `identity:get_user(user_id=6bca01c7266944329ecb3cbe0887bc8e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:12.201899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111983c-5840-4183-9074-0b83085047b6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:12.203693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 86/174] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:12 2022] GET /identity/v3/users/6bca01c7266944329ecb3cbe0887bc8e => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:12.215452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111983c-5840-4183-9074-0b83085047b6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:12.246374 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:12.280293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757b663-4722-455a-a3dd-90d2131dc18d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:12.331504 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdeeb92-dd3d-40ac-a8b9-18733a021252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:12.332518 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdeeb92-dd3d-40ac-a8b9-18733a021252 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:12.332972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdeeb92-dd3d-40ac-a8b9-18733a021252 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:12.333387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdeeb92-dd3d-40ac-a8b9-18733a021252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:12.411709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdeeb92-dd3d-40ac-a8b9-18733a021252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:12.413980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdeeb92-dd3d-40ac-a8b9-18733a021252 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:12.514137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:12.519183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111983c-5840-4183-9074-0b83085047b6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:12.580500 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d9ae3-e822-4348-97e3-4332933369e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:12.581151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d9ae3-e822-4348-97e3-4332933369e5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:12.581431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d9ae3-e822-4348-97e3-4332933369e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:12.581701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d9ae3-e822-4348-97e3-4332933369e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:12.588857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 89/175] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:12.594622 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdeeb92-dd3d-40ac-a8b9-18733a021252 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:12.692147 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:12.695163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d9ae3-e822-4348-97e3-4332933369e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:12.696875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d9ae3-e822-4348-97e3-4332933369e5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:12.765529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdeeb92-dd3d-40ac-a8b9-18733a021252 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:12.826451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5e7e0-21c6-41e8-b176-f96e0d3e7118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:12.827590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5e7e0-21c6-41e8-b176-f96e0d3e7118 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:12.828030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5e7e0-21c6-41e8-b176-f96e0d3e7118 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:12.828450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5e7e0-21c6-41e8-b176-f96e0d3e7118 None admin] PATH_INFO: `/v3/projects/040fe7f7a2fe4f9795fdc754be42c648` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:12.831736 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5e7e0-21c6-41e8-b176-f96e0d3e7118 None admin] RBAC: Authorizing `identity:get_project(project_id=040fe7f7a2fe4f9795fdc754be42c648)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:12.833353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5e7e0-21c6-41e8-b176-f96e0d3e7118 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:12.836592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 90/176] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:12 2022] GET /identity/v3/projects/040fe7f7a2fe4f9795fdc754be42c648 => generated 312 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:12.842188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5e7e0-21c6-41e8-b176-f96e0d3e7118 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:12.886631 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 87/177] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:12.890467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:12.892289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d9ae3-e822-4348-97e3-4332933369e5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:13.006525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5e7e0-21c6-41e8-b176-f96e0d3e7118 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:13.070703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ce0fa-d941-4979-8d23-b4c13e6f6442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:13.072078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ce0fa-d941-4979-8d23-b4c13e6f6442 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:13.072515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ce0fa-d941-4979-8d23-b4c13e6f6442 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:13.072962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ce0fa-d941-4979-8d23-b4c13e6f6442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:13.074050 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ce0fa-d941-4979-8d23-b4c13e6f6442 None admin] RBAC: Adding query filter params (role.id=86e07561a0854fc28db2e9f357e53d66, scope.project.id=040fe7f7a2fe4f9795fdc754be42c648, user.id=9b82c4a811bc44ebab8a6b00f9536634) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:13.074528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ce0fa-d941-4979-8d23-b4c13e6f6442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:13.076687 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ce0fa-d941-4979-8d23-b4c13e6f6442 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:13.090946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 91/178] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:12 2022] GET /identity/v3/role_assignments?user.id=9b82c4a811bc44ebab8a6b00f9536634&scope.project.id=040fe7f7a2fe4f9795fdc754be42c648&role.id=86e07561a0854fc28db2e9f357e53d66 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:13.111888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:13.115422 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d9ae3-e822-4348-97e3-4332933369e5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:13.158368 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3952b3b-3861-4af8-a75e-5fb9077dcc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:13.159494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3952b3b-3861-4af8-a75e-5fb9077dcc61 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:13.159986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3952b3b-3861-4af8-a75e-5fb9077dcc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:13.160440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3952b3b-3861-4af8-a75e-5fb9077dcc61 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:13.166714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3952b3b-3861-4af8-a75e-5fb9077dcc61 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:13.168963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3952b3b-3861-4af8-a75e-5fb9077dcc61 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:13.171345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 88/179] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:12 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:13.177049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ce0fa-d941-4979-8d23-b4c13e6f6442 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:13.291472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bf935-cc9e-492e-9f98-d381ee7644c0 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:13.292095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bf935-cc9e-492e-9f98-d381ee7644c0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:13.292626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bf935-cc9e-492e-9f98-d381ee7644c0 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:13.295275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 89/180] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:13.302629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b8553-9128-47e8-8298-63e60995a69d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:13.303236 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b8553-9128-47e8-8298-63e60995a69d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:13.303728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b8553-9128-47e8-8298-63e60995a69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:13.315208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:13.326647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6e6b8553-9128-47e8-8298-63e60995a69d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:13.328197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6e6b8553-9128-47e8-8298-63e60995a69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:13.328963 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ce0fa-d941-4979-8d23-b4c13e6f6442 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:13.377951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ca162-5b13-4777-b79e-81b0768a5ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:13.378690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ca162-5b13-4777-b79e-81b0768a5ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:13.379085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ca162-5b13-4777-b79e-81b0768a5ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:13.379371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ca162-5b13-4777-b79e-81b0768a5ec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:13.380342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ca162-5b13-4777-b79e-81b0768a5ec0 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1, user.id=6bca01c7266944329ecb3cbe0887bc8e) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:13.380661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ca162-5b13-4777-b79e-81b0768a5ec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:13.382217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ca162-5b13-4777-b79e-81b0768a5ec0 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:13.393766 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 92/181] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:13 2022] GET /identity/v3/role_assignments?user.id=6bca01c7266944329ecb3cbe0887bc8e&scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:13.433350 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 90/182] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:13.469278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ca162-5b13-4777-b79e-81b0768a5ec0 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:13.596284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ca162-5b13-4777-b79e-81b0768a5ec0 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:13.642240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7dd7f-443a-4e10-9b7f-e35dc8117ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:13.643044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7dd7f-443a-4e10-9b7f-e35dc8117ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:13.643328 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7dd7f-443a-4e10-9b7f-e35dc8117ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:13.643602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7dd7f-443a-4e10-9b7f-e35dc8117ff4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:13.650588 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cd6fa-823a-474e-a661-2c67757f32f8 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:13.651465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cd6fa-823a-474e-a661-2c67757f32f8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:13.652585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cd6fa-823a-474e-a661-2c67757f32f8 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:13.654211 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 91/183] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:13.657293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7dd7f-443a-4e10-9b7f-e35dc8117ff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:13.660337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6a7c7-12b4-4429-80c0-8078a9d76d5d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:13.660872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6a7c7-12b4-4429-80c0-8078a9d76d5d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:13.661347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6a7c7-12b4-4429-80c0-8078a9d76d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-5df7dd7f-443a-4e10-9b7f-e35dc8117ff4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:13.665323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:13.666278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:13.667048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 93/184] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:26:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:13.675331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7dd7f-443a-4e10-9b7f-e35dc8117ff4 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:13.687373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c1d6a7c7-12b4-4429-80c0-8078a9d76d5d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:13.688941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c1d6a7c7-12b4-4429-80c0-8078a9d76d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:13.703458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:13.779551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 92/185] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:13.780706 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121c846-d965-4a7d-b7ac-ca3e57f27c9a None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:13.781115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121c846-d965-4a7d-b7ac-ca3e57f27c9a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:13.781655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121c846-d965-4a7d-b7ac-ca3e57f27c9a None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:13.783112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 93/186] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:13.789793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15263a16-54aa-4e9d-8076-d4cbbe74151a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:13.790693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15263a16-54aa-4e9d-8076-d4cbbe74151a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:13.791205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15263a16-54aa-4e9d-8076-d4cbbe74151a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:13.813469 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7dd7f-443a-4e10-9b7f-e35dc8117ff4 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:13.817262 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-15263a16-54aa-4e9d-8076-d4cbbe74151a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:13.818474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-15263a16-54aa-4e9d-8076-d4cbbe74151a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:13.847190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:13.852246 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a2e60-3602-45b6-9142-eb768d0b8057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:13.853001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a2e60-3602-45b6-9142-eb768d0b8057 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:13.853287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a2e60-3602-45b6-9142-eb768d0b8057 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:13.853564 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a2e60-3602-45b6-9142-eb768d0b8057 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:13.854370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a2e60-3602-45b6-9142-eb768d0b8057 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:13.854676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a2e60-3602-45b6-9142-eb768d0b8057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:13.856448 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a2e60-3602-45b6-9142-eb768d0b8057 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:13.864908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 94/187] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:13.866920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a2e60-3602-45b6-9142-eb768d0b8057 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:13.899885 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 94/188] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:13.906982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15263a16-54aa-4e9d-8076-d4cbbe74151a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:13.952433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:13.977032 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a2e60-3602-45b6-9142-eb768d0b8057 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:14.010156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15263a16-54aa-4e9d-8076-d4cbbe74151a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:14.011034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b03b7-6218-4ba9-b579-6ab31718666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:14.011850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b03b7-6218-4ba9-b579-6ab31718666f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.012134 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b03b7-6218-4ba9-b579-6ab31718666f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.012418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b03b7-6218-4ba9-b579-6ab31718666f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.023910 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b03b7-6218-4ba9-b579-6ab31718666f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-db3b03b7-6218-4ba9-b579-6ab31718666f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:14.032566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:14.033427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:14.034232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 95/189] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:26:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:14.034865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b03b7-6218-4ba9-b579-6ab31718666f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:14.046243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22df9ea-0894-49eb-b296-eaa9493d428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:14.047125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22df9ea-0894-49eb-b296-eaa9493d428b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.047410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22df9ea-0894-49eb-b296-eaa9493d428b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.047684 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22df9ea-0894-49eb-b296-eaa9493d428b None admin] PATH_INFO: `/v3/users/4c5a1f97dcc74239afdf377346a19a5a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.063085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22df9ea-0894-49eb-b296-eaa9493d428b None admin] RBAC: Authorizing `identity:get_user(user_id=4c5a1f97dcc74239afdf377346a19a5a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:14.064592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22df9ea-0894-49eb-b296-eaa9493d428b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:14.066325 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 95/190] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:13 2022] GET /identity/v3/users/4c5a1f97dcc74239afdf377346a19a5a => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:14.071799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e050d-c8cf-4858-834e-cbdc710f29fd None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.072326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e050d-c8cf-4858-834e-cbdc710f29fd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.073103 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e050d-c8cf-4858-834e-cbdc710f29fd None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.077975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 96/191] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:14.077975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e050d-c8cf-4858-834e-cbdc710f29fd None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:14.100723 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:14.144557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b03b7-6218-4ba9-b579-6ab31718666f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:14.185644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1760fc-b147-4ec0-9827-2b39d687630f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:14.186483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1760fc-b147-4ec0-9827-2b39d687630f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.186767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1760fc-b147-4ec0-9827-2b39d687630f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.187032 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1760fc-b147-4ec0-9827-2b39d687630f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.194362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:14.196034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e050d-c8cf-4858-834e-cbdc710f29fd None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:14.200477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1760fc-b147-4ec0-9827-2b39d687630f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-3a1760fc-b147-4ec0-9827-2b39d687630f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:14.208616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:14.209496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:14.210346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 96/192] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:14.211031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1760fc-b147-4ec0-9827-2b39d687630f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:14.237726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f1ddf-1dcf-4266-8108-a5ccc9ca7b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:14.238667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f1ddf-1dcf-4266-8108-a5ccc9ca7b0f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.239143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f1ddf-1dcf-4266-8108-a5ccc9ca7b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.239619 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f1ddf-1dcf-4266-8108-a5ccc9ca7b0f None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.240570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f1ddf-1dcf-4266-8108-a5ccc9ca7b0f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:14.241055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f1ddf-1dcf-4266-8108-a5ccc9ca7b0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:14.242654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f1ddf-1dcf-4266-8108-a5ccc9ca7b0f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:14.251041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 97/193] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:14.252102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95070155-ecc0-4851-a61c-eecb7c202dee None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.252392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95070155-ecc0-4851-a61c-eecb7c202dee None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.252662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95070155-ecc0-4851-a61c-eecb7c202dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.271154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-95070155-ecc0-4851-a61c-eecb7c202dee None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:14.272475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-95070155-ecc0-4851-a61c-eecb7c202dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:14.312987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1760fc-b147-4ec0-9827-2b39d687630f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:14.346972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 98/194] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:14.347825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95070155-ecc0-4851-a61c-eecb7c202dee None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:14.348241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804f8e2-fa68-4b80-bb60-07bbc174bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:14.348833 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804f8e2-fa68-4b80-bb60-07bbc174bee8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.349120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804f8e2-fa68-4b80-bb60-07bbc174bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.349398 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804f8e2-fa68-4b80-bb60-07bbc174bee8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.359200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:14.423498 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:14.427342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804f8e2-fa68-4b80-bb60-07bbc174bee8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:14.428984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804f8e2-fa68-4b80-bb60-07bbc174bee8 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:14.457505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95070155-ecc0-4851-a61c-eecb7c202dee None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:14.490542 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6781f-a16a-4c36-8c91-521d7a05f665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:14.491199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6781f-a16a-4c36-8c91-521d7a05f665 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.491481 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6781f-a16a-4c36-8c91-521d7a05f665 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.491753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6781f-a16a-4c36-8c91-521d7a05f665 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.492492 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6781f-a16a-4c36-8c91-521d7a05f665 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:14.492800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6781f-a16a-4c36-8c91-521d7a05f665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:14.494201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6781f-a16a-4c36-8c91-521d7a05f665 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:14.502470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 99/195] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:14.506077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6781f-a16a-4c36-8c91-521d7a05f665 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:14.529949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 97/196] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:14.530754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804f8e2-fa68-4b80-bb60-07bbc174bee8 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:14.577829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:14.608959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6781f-a16a-4c36-8c91-521d7a05f665 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:14.622168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:14.625635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804f8e2-fa68-4b80-bb60-07bbc174bee8 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:14.642234 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f2514-4b73-4dc5-bd4a-c7f5f820b8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:14.643098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f2514-4b73-4dc5-bd4a-c7f5f820b8ac None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.643381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f2514-4b73-4dc5-bd4a-c7f5f820b8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.643653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f2514-4b73-4dc5-bd4a-c7f5f820b8ac None admin] PATH_INFO: `/v3/users/4c5a1f97dcc74239afdf377346a19a5a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.644838 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f2514-4b73-4dc5-bd4a-c7f5f820b8ac None admin] RBAC: Authorizing `identity:get_user(user_id=4c5a1f97dcc74239afdf377346a19a5a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:14.646310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f2514-4b73-4dc5-bd4a-c7f5f820b8ac None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:14.647950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 100/197] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:14 2022] GET /identity/v3/users/4c5a1f97dcc74239afdf377346a19a5a => generated 283 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:14.650944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f2514-4b73-4dc5-bd4a-c7f5f820b8ac None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:14.659135 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef109af-3b7a-4807-8688-e0e4bda82263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:14.659928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef109af-3b7a-4807-8688-e0e4bda82263 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.660213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef109af-3b7a-4807-8688-e0e4bda82263 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.660487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef109af-3b7a-4807-8688-e0e4bda82263 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.661655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef109af-3b7a-4807-8688-e0e4bda82263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:14.663122 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef109af-3b7a-4807-8688-e0e4bda82263 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:14.664855 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 98/198] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:14.665464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef109af-3b7a-4807-8688-e0e4bda82263 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:14.751421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f2514-4b73-4dc5-bd4a-c7f5f820b8ac None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:14.762191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef109af-3b7a-4807-8688-e0e4bda82263 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:14.783983 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28937692-f9ce-48ac-a163-7a8e43b21626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:14.784848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28937692-f9ce-48ac-a163-7a8e43b21626 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.785129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28937692-f9ce-48ac-a163-7a8e43b21626 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.785406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28937692-f9ce-48ac-a163-7a8e43b21626 None admin] PATH_INFO: `/v3/users/bc0dcabcdc0241759d773e360b0cb0d6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.786610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28937692-f9ce-48ac-a163-7a8e43b21626 None admin] RBAC: Authorizing `identity:get_user(user_id=bc0dcabcdc0241759d773e360b0cb0d6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:14.788044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28937692-f9ce-48ac-a163-7a8e43b21626 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:14.789681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 101/199] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:14 2022] GET /identity/v3/users/bc0dcabcdc0241759d773e360b0cb0d6 => generated 289 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:14.789681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28937692-f9ce-48ac-a163-7a8e43b21626 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:14.799346 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:14.801317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abbf41-c8a8-415a-a6a0-de5bd5877e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:14.802319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abbf41-c8a8-415a-a6a0-de5bd5877e41 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.802603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abbf41-c8a8-415a-a6a0-de5bd5877e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.802881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abbf41-c8a8-415a-a6a0-de5bd5877e41 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.804082 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abbf41-c8a8-415a-a6a0-de5bd5877e41 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:14.805521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abbf41-c8a8-415a-a6a0-de5bd5877e41 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:14.807272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 99/200] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:14 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:14.808509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abbf41-c8a8-415a-a6a0-de5bd5877e41 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:14.842619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:14.889433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28937692-f9ce-48ac-a163-7a8e43b21626 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:14.907182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abbf41-c8a8-415a-a6a0-de5bd5877e41 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:14.923731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002809a3-d28b-42ed-95c6-2188cdf83279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:14.924373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002809a3-d28b-42ed-95c6-2188cdf83279 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.924651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002809a3-d28b-42ed-95c6-2188cdf83279 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.924928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002809a3-d28b-42ed-95c6-2188cdf83279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.940753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26cf3c-ade6-4fb1-8e46-223370490477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:14.941412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26cf3c-ade6-4fb1-8e46-223370490477 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:14.941697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26cf3c-ade6-4fb1-8e46-223370490477 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:14.941999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26cf3c-ade6-4fb1-8e46-223370490477 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:14.942712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26cf3c-ade6-4fb1-8e46-223370490477 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:14.944142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26cf3c-ade6-4fb1-8e46-223370490477 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:14.965237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 100/201] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:26:14 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:14.965237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26cf3c-ade6-4fb1-8e46-223370490477 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:14.993216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002809a3-d28b-42ed-95c6-2188cdf83279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:14.994750 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002809a3-d28b-42ed-95c6-2188cdf83279 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:15.004868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:15.076067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:15.088520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26cf3c-ade6-4fb1-8e46-223370490477 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:15.113089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 102/202] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:15.113089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002809a3-d28b-42ed-95c6-2188cdf83279 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:15.125754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf0bbf-ec98-481a-982c-7036bd9afe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:15.126434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf0bbf-ec98-481a-982c-7036bd9afe8a None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:15.126719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf0bbf-ec98-481a-982c-7036bd9afe8a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:15.126993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf0bbf-ec98-481a-982c-7036bd9afe8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:15.209536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf0bbf-ec98-481a-982c-7036bd9afe8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:15.211292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf0bbf-ec98-481a-982c-7036bd9afe8a None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:15.240761 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002809a3-d28b-42ed-95c6-2188cdf83279 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:15.250202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:15.285932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f0aaa-e822-4b2e-880d-93e9099f28a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:15.286612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f0aaa-e822-4b2e-880d-93e9099f28a4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:15.286891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f0aaa-e822-4b2e-880d-93e9099f28a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:15.287162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f0aaa-e822-4b2e-880d-93e9099f28a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:15.288003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f0aaa-e822-4b2e-880d-93e9099f28a4 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e, user.id=4c5a1f97dcc74239afdf377346a19a5a) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:15.288311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f0aaa-e822-4b2e-880d-93e9099f28a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:15.289784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f0aaa-e822-4b2e-880d-93e9099f28a4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:15.296037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 103/203] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:15 2022] GET /identity/v3/role_assignments?user.id=4c5a1f97dcc74239afdf377346a19a5a&scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:15.299201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f0aaa-e822-4b2e-880d-93e9099f28a4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:15.335943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:15.349829 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 101/204] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:15.358988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf0bbf-ec98-481a-982c-7036bd9afe8a None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:15.432888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f0aaa-e822-4b2e-880d-93e9099f28a4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:15.482727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99c7d2-8d41-434c-a521-b9063bd6471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:15.483528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99c7d2-8d41-434c-a521-b9063bd6471c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:15.483805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99c7d2-8d41-434c-a521-b9063bd6471c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:15.484074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99c7d2-8d41-434c-a521-b9063bd6471c None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:15.485853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99c7d2-8d41-434c-a521-b9063bd6471c None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:15.487348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99c7d2-8d41-434c-a521-b9063bd6471c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:15.497357 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 104/205] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:15 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:15.498364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99c7d2-8d41-434c-a521-b9063bd6471c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:15.574824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:15.603696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf0bbf-ec98-481a-982c-7036bd9afe8a None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:15.676526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a273fbf-527e-446b-b60b-f1a192c0df02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:15.677394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a273fbf-527e-446b-b60b-f1a192c0df02 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:15.677676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a273fbf-527e-446b-b60b-f1a192c0df02 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:15.677968 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a273fbf-527e-446b-b60b-f1a192c0df02 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:15.684592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a273fbf-527e-446b-b60b-f1a192c0df02 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:15.684592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a273fbf-527e-446b-b60b-f1a192c0df02 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:15.686791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 102/206] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:15 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:15.692106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a273fbf-527e-446b-b60b-f1a192c0df02 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:15.706875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99c7d2-8d41-434c-a521-b9063bd6471c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:15.778996 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce177a-13b9-49f0-b389-13f5fc6b13eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:15.781475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce177a-13b9-49f0-b389-13f5fc6b13eb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:15.781976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce177a-13b9-49f0-b389-13f5fc6b13eb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:15.782447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce177a-13b9-49f0-b389-13f5fc6b13eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:15.785493 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce177a-13b9-49f0-b389-13f5fc6b13eb None admin] RBAC: Adding query filter params (role.id=950510c081de4ab6b12f383c0748450d, scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e, user.id=4c5a1f97dcc74239afdf377346a19a5a) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:15.786030 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce177a-13b9-49f0-b389-13f5fc6b13eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:15.787927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce177a-13b9-49f0-b389-13f5fc6b13eb None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:15.805097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 105/207] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:15 2022] GET /identity/v3/role_assignments?user.id=4c5a1f97dcc74239afdf377346a19a5a&scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e&role.id=950510c081de4ab6b12f383c0748450d => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:15.814416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:15.862217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a273fbf-527e-446b-b60b-f1a192c0df02 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:15.899324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41462303-c2a1-4846-9b3c-c6f1198d0e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:15.900352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41462303-c2a1-4846-9b3c-c6f1198d0e76 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:15.900817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41462303-c2a1-4846-9b3c-c6f1198d0e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:15.901270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41462303-c2a1-4846-9b3c-c6f1198d0e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:15.904864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41462303-c2a1-4846-9b3c-c6f1198d0e76 None admin] RBAC: Adding query filter params (role.id=06b987ffe3ed42d1bc1963d1459eaaa6, scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1, user.id=bc0dcabcdc0241759d773e360b0cb0d6) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:15.905410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41462303-c2a1-4846-9b3c-c6f1198d0e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:15.907126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41462303-c2a1-4846-9b3c-c6f1198d0e76 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:15.914724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 103/208] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:15 2022] GET /identity/v3/role_assignments?user.id=bc0dcabcdc0241759d773e360b0cb0d6&scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1&role.id=06b987ffe3ed42d1bc1963d1459eaaa6 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:17.168939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd7dc0-0186-4bff-a678-b9d4b633777e None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:17.169441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd7dc0-0186-4bff-a678-b9d4b633777e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:17.169555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd7dc0-0186-4bff-a678-b9d4b633777e None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:17.171109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 106/209] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:17.176470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b887ac-3a95-4000-a9ab-6bc6d452e7d8 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:17.176857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b887ac-3a95-4000-a9ab-6bc6d452e7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:17.177042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b887ac-3a95-4000-a9ab-6bc6d452e7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:17.196769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-c5b887ac-3a95-4000-a9ab-6bc6d452e7d8 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:17.197980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c5b887ac-3a95-4000-a9ab-6bc6d452e7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:17.268657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 104/210] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:17.290990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd7dc0-0186-4bff-a678-b9d4b633777e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:17.374099 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdb3c1-5444-47f0-909d-80160144a556 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:17.374441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdb3c1-5444-47f0-909d-80160144a556 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:17.374650 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdb3c1-5444-47f0-909d-80160144a556 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:17.376070 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 105/211] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:17.381419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49317e3-7025-4216-8be3-e9f1e1d83fd1 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:17.381710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49317e3-7025-4216-8be3-e9f1e1d83fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:17.382012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49317e3-7025-4216-8be3-e9f1e1d83fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:17.404139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-d49317e3-7025-4216-8be3-e9f1e1d83fd1 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:17.404976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd7dc0-0186-4bff-a678-b9d4b633777e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:17.405489 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d49317e3-7025-4216-8be3-e9f1e1d83fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:17.406725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:17.449126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68d4e1-67d3-4650-984e-018487acef8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:17.450369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68d4e1-67d3-4650-984e-018487acef8a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:17.451025 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68d4e1-67d3-4650-984e-018487acef8a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:17.451935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68d4e1-67d3-4650-984e-018487acef8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:17.470339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68d4e1-67d3-4650-984e-018487acef8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:17.471591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-3a68d4e1-67d3-4650-984e-018487acef8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:17.484398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:17.485462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:17.486415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:17.486415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:17.486415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:17.486415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:17.486415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:17.486415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:17.486415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:17.486415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:17.486415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:17.486415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:17.486415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:17.486415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:17.489748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 107/212] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:17.496033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68d4e1-67d3-4650-984e-018487acef8a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:17.502359 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 106/213] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:17.532565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49317e3-7025-4216-8be3-e9f1e1d83fd1 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:17.629267 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68d4e1-67d3-4650-984e-018487acef8a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:17.654877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49317e3-7025-4216-8be3-e9f1e1d83fd1 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:17.666658 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc5dbf-47f4-49d1-9507-4ba5661f5c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:17.667903 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc5dbf-47f4-49d1-9507-4ba5661f5c56 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:17.668216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc5dbf-47f4-49d1-9507-4ba5661f5c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:17.668499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc5dbf-47f4-49d1-9507-4ba5661f5c56 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:17.669250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc5dbf-47f4-49d1-9507-4ba5661f5c56 None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:17.669586 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc5dbf-47f4-49d1-9507-4ba5661f5c56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:17.671012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc5dbf-47f4-49d1-9507-4ba5661f5c56 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:17.681399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 108/214] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:17.682735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef7f7a-4818-470a-814f-d97c76e0e750 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:17.683046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef7f7a-4818-470a-814f-d97c76e0e750 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:17.683319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef7f7a-4818-470a-814f-d97c76e0e750 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:17.688739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 109/215] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:17.690150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85520175-aa9b-41b0-9897-6a8481f49674 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:17.690433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85520175-aa9b-41b0-9897-6a8481f49674 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:17.690706 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85520175-aa9b-41b0-9897-6a8481f49674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:17.704681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcf2ec-6a83-4621-a569-14a6bdea08cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:17.705571 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcf2ec-6a83-4621-a569-14a6bdea08cb None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:17.705854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcf2ec-6a83-4621-a569-14a6bdea08cb None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:17.706156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcf2ec-6a83-4621-a569-14a6bdea08cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:17.710733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-85520175-aa9b-41b0-9897-6a8481f49674 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:17.711078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcf2ec-6a83-4621-a569-14a6bdea08cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:17.712003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-85520175-aa9b-41b0-9897-6a8481f49674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:17.712654 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcf2ec-6a83-4621-a569-14a6bdea08cb None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:17.715161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 107/216] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:17.715720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcf2ec-6a83-4621-a569-14a6bdea08cb None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:17.737543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:17.774393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:17.799646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 110/217] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:17.800479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85520175-aa9b-41b0-9897-6a8481f49674 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:17.840451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcf2ec-6a83-4621-a569-14a6bdea08cb None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:17.876756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4554b-3e31-461c-a7f3-0ee303047d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:17.877655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4554b-3e31-461c-a7f3-0ee303047d73 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:17.877934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4554b-3e31-461c-a7f3-0ee303047d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:17.878209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4554b-3e31-461c-a7f3-0ee303047d73 None admin] PATH_INFO: `/v3/users/3b027f09d3274d9ea581af6d66e8afae` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:17.894115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4554b-3e31-461c-a7f3-0ee303047d73 None admin] RBAC: Authorizing `identity:get_user(user_id=3b027f09d3274d9ea581af6d66e8afae)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:17.895967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4554b-3e31-461c-a7f3-0ee303047d73 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:17.897991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 108/218] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:17 2022] GET /identity/v3/users/3b027f09d3274d9ea581af6d66e8afae => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:17.898526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4554b-3e31-461c-a7f3-0ee303047d73 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:17.944299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85520175-aa9b-41b0-9897-6a8481f49674 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:17.989528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ab93f-a5ec-435b-ab54-ea38ceda82e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:17.990151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ab93f-a5ec-435b-ab54-ea38ceda82e3 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:17.990437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ab93f-a5ec-435b-ab54-ea38ceda82e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:17.990715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ab93f-a5ec-435b-ab54-ea38ceda82e3 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:17.991449 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ab93f-a5ec-435b-ab54-ea38ceda82e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:17.992887 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ab93f-a5ec-435b-ab54-ea38ceda82e3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:18.019066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 111/219] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:26:17 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:18.019066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ab93f-a5ec-435b-ab54-ea38ceda82e3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:18.021552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4554b-3e31-461c-a7f3-0ee303047d73 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:18.037257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:18.046813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:18.074264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d09f77-ba38-41c2-8f48-04abb8eac966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:18.075109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d09f77-ba38-41c2-8f48-04abb8eac966 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:18.075391 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d09f77-ba38-41c2-8f48-04abb8eac966 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:18.075667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d09f77-ba38-41c2-8f48-04abb8eac966 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:18.090293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d09f77-ba38-41c2-8f48-04abb8eac966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-e8d09f77-ba38-41c2-8f48-04abb8eac966 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:18.100680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:18.101638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:18.102477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 109/220] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:26:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:18.108698 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d09f77-ba38-41c2-8f48-04abb8eac966 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:18.160955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ab93f-a5ec-435b-ab54-ea38ceda82e3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:18.207851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e42f9-19cf-4e04-90ef-7777aca52594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:18.209369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e42f9-19cf-4e04-90ef-7777aca52594 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:18.209556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e42f9-19cf-4e04-90ef-7777aca52594 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:18.210689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e42f9-19cf-4e04-90ef-7777aca52594 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:18.237382 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d09f77-ba38-41c2-8f48-04abb8eac966 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:18.285228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d36964-d15d-4f32-bbf4-167828bffb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:18.286149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d36964-d15d-4f32-bbf4-167828bffb81 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:18.286433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d36964-d15d-4f32-bbf4-167828bffb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:18.286710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d36964-d15d-4f32-bbf4-167828bffb81 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:18.287470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d36964-d15d-4f32-bbf4-167828bffb81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:18.287776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d36964-d15d-4f32-bbf4-167828bffb81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:18.289768 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d36964-d15d-4f32-bbf4-167828bffb81 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:18.296672 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e42f9-19cf-4e04-90ef-7777aca52594 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:18.298518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e42f9-19cf-4e04-90ef-7777aca52594 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:18.299711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 110/221] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:18.300828 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603051cc-bb22-4ea7-b430-83c3054c9102 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:18.301117 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603051cc-bb22-4ea7-b430-83c3054c9102 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:18.301393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603051cc-bb22-4ea7-b430-83c3054c9102 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:18.302998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 111/222] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:18.309840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d40c9-80e8-4d91-8ae7-f06f45439da2 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:18.310234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d40c9-80e8-4d91-8ae7-f06f45439da2 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:18.310519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d40c9-80e8-4d91-8ae7-f06f45439da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:18.316187 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:18.333223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f40d40c9-80e8-4d91-8ae7-f06f45439da2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:18.334458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f40d40c9-80e8-4d91-8ae7-f06f45439da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:18.367143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:18.415745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 112/224] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:18.415928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 112/224] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:18.416608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d40c9-80e8-4d91-8ae7-f06f45439da2 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:18.431243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e42f9-19cf-4e04-90ef-7777aca52594 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:18.533153 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e42f9-19cf-4e04-90ef-7777aca52594 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:18.533980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d40c9-80e8-4d91-8ae7-f06f45439da2 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:18.550288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:18.571207 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6684ea-0bcd-448e-a78b-565572e80c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:18.572126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6684ea-0bcd-448e-a78b-565572e80c27 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:18.572657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6684ea-0bcd-448e-a78b-565572e80c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:18.573273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6684ea-0bcd-448e-a78b-565572e80c27 None admin] PATH_INFO: `/v3/users/4c5a1f97dcc74239afdf377346a19a5a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:18.575102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9903173-e92d-40f8-a841-2076a3c4699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:18.575670 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6684ea-0bcd-448e-a78b-565572e80c27 None admin] RBAC: Authorizing `identity:get_user(user_id=4c5a1f97dcc74239afdf377346a19a5a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:18.576443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9903173-e92d-40f8-a841-2076a3c4699e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:18.576915 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9903173-e92d-40f8-a841-2076a3c4699e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:18.577145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6684ea-0bcd-448e-a78b-565572e80c27 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:18.577571 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9903173-e92d-40f8-a841-2076a3c4699e None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:18.579006 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 113/225] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:18 2022] GET /identity/v3/users/4c5a1f97dcc74239afdf377346a19a5a => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:18.579493 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9903173-e92d-40f8-a841-2076a3c4699e None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:18.579594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6684ea-0bcd-448e-a78b-565572e80c27 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:18.581874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9903173-e92d-40f8-a841-2076a3c4699e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:18.585537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 113/226] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:18 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:18.586606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9903173-e92d-40f8-a841-2076a3c4699e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:18.609341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:18.695697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6684ea-0bcd-448e-a78b-565572e80c27 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:18.707287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9903173-e92d-40f8-a841-2076a3c4699e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:18.741107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef866dd5-0bba-4fd7-a2b3-93ecacc1a1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:18.742227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef866dd5-0bba-4fd7-a2b3-93ecacc1a1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:18.742515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef866dd5-0bba-4fd7-a2b3-93ecacc1a1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:18.742798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef866dd5-0bba-4fd7-a2b3-93ecacc1a1c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:18.746024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cedce-74b5-431c-9301-75d5b91ac135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:18.746989 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cedce-74b5-431c-9301-75d5b91ac135 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:18.747374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cedce-74b5-431c-9301-75d5b91ac135 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:18.747748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cedce-74b5-431c-9301-75d5b91ac135 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:18.749260 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cedce-74b5-431c-9301-75d5b91ac135 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:18.751121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cedce-74b5-431c-9301-75d5b91ac135 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:18.753333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef866dd5-0bba-4fd7-a2b3-93ecacc1a1c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:18.753454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 114/227] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:18 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:18.754347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cedce-74b5-431c-9301-75d5b91ac135 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-ef866dd5-0bba-4fd7-a2b3-93ecacc1a1c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:18.764282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:18.765163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:18.766129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:18.766129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:18.766129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:18.766129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:18.766129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:18.766129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:18.766129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:18.766129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:18.766129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:18.766129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:18.766129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:18.766129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:18.770352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 114/228] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:26:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:18.771103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef866dd5-0bba-4fd7-a2b3-93ecacc1a1c8 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:18.814577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:18.870241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:18.886839 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cedce-74b5-431c-9301-75d5b91ac135 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:18.890773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef866dd5-0bba-4fd7-a2b3-93ecacc1a1c8 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:18.925495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761cf4-f65a-46c6-baea-13d8fabf0739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:18.926207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761cf4-f65a-46c6-baea-13d8fabf0739 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:18.926496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761cf4-f65a-46c6-baea-13d8fabf0739 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:18.926776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761cf4-f65a-46c6-baea-13d8fabf0739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:18.927673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a761cf4-f65a-46c6-baea-13d8fabf0739 None admin] RBAC: Adding query filter params (role.id=86e07561a0854fc28db2e9f357e53d66, scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e, user.id=3b027f09d3274d9ea581af6d66e8afae) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:18.928092 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a761cf4-f65a-46c6-baea-13d8fabf0739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:18.929498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a761cf4-f65a-46c6-baea-13d8fabf0739 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:18.935468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 115/229] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:18 2022] GET /identity/v3/role_assignments?user.id=3b027f09d3274d9ea581af6d66e8afae&scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e&role.id=86e07561a0854fc28db2e9f357e53d66 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:18.940529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13d7e7-7a84-4626-ab5a-b65c6c7bcef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:18.941753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13d7e7-7a84-4626-ab5a-b65c6c7bcef6 None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:18.942357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13d7e7-7a84-4626-ab5a-b65c6c7bcef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:18.942839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13d7e7-7a84-4626-ab5a-b65c6c7bcef6 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e/users/4c5a1f97dcc74239afdf377346a19a5a/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:18.945341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13d7e7-7a84-4626-ab5a-b65c6c7bcef6 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb4447b59c0746648ea2c2d04f5e3c7e, user_id=4c5a1f97dcc74239afdf377346a19a5a, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:18.946234 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761cf4-f65a-46c6-baea-13d8fabf0739 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:18.947485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13d7e7-7a84-4626-ab5a-b65c6c7bcef6 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:18.968011 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 115/230] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:26:18 2022] PUT /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e/users/4c5a1f97dcc74239afdf377346a19a5a/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:19.116376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761cf4-f65a-46c6-baea-13d8fabf0739 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:19.131740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:19.161960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0ccf0-4aa8-4ef9-b10a-1fab6b9f5e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:19.162650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0ccf0-4aa8-4ef9-b10a-1fab6b9f5e9a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:19.162938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0ccf0-4aa8-4ef9-b10a-1fab6b9f5e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:19.163218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0ccf0-4aa8-4ef9-b10a-1fab6b9f5e9a None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:19.163985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0ccf0-4aa8-4ef9-b10a-1fab6b9f5e9a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:19.164294 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0ccf0-4aa8-4ef9-b10a-1fab6b9f5e9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:19.165685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0ccf0-4aa8-4ef9-b10a-1fab6b9f5e9a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:19.174971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 116/231] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:19.180715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13d7e7-7a84-4626-ab5a-b65c6c7bcef6 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:19.291713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13d7e7-7a84-4626-ab5a-b65c6c7bcef6 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:19.328093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1c56c-a402-4132-bc7c-bf2d30f0a0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:19.328993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1c56c-a402-4132-bc7c-bf2d30f0a0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:19.329277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1c56c-a402-4132-bc7c-bf2d30f0a0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:19.329556 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1c56c-a402-4132-bc7c-bf2d30f0a0e7 None admin] PATH_INFO: `/v3/users/4c5a1f97dcc74239afdf377346a19a5a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:19.330865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1c56c-a402-4132-bc7c-bf2d30f0a0e7 None admin] RBAC: Authorizing `identity:get_user(user_id=4c5a1f97dcc74239afdf377346a19a5a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:19.332401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1c56c-a402-4132-bc7c-bf2d30f0a0e7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:19.334209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 116/232] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:19 2022] GET /identity/v3/users/4c5a1f97dcc74239afdf377346a19a5a => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:19.340310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0ccf0-4aa8-4ef9-b10a-1fab6b9f5e9a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:19.470415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0ccf0-4aa8-4ef9-b10a-1fab6b9f5e9a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:19.516500 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93394fd-140c-4108-983f-d11d7d605131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:19.517376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93394fd-140c-4108-983f-d11d7d605131 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:19.517660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93394fd-140c-4108-983f-d11d7d605131 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:19.518025 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93394fd-140c-4108-983f-d11d7d605131 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:19.522634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93394fd-140c-4108-983f-d11d7d605131 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:19.524114 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93394fd-140c-4108-983f-d11d7d605131 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:19.526021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 117/233] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:19 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:19.531694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1c56c-a402-4132-bc7c-bf2d30f0a0e7 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:19.551342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:19.649660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1c56c-a402-4132-bc7c-bf2d30f0a0e7 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:19.684180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0c341-b30e-441f-b802-450d825bff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:19.685082 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0c341-b30e-441f-b802-450d825bff7d None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:19.685365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0c341-b30e-441f-b802-450d825bff7d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:19.685642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0c341-b30e-441f-b802-450d825bff7d None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e/users/4c5a1f97dcc74239afdf377346a19a5a/roles/950510c081de4ab6b12f383c0748450d` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:19.688001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0c341-b30e-441f-b802-450d825bff7d None admin] RBAC: Authorizing `identity:create_grant(project_id=eb4447b59c0746648ea2c2d04f5e3c7e, user_id=4c5a1f97dcc74239afdf377346a19a5a, role_id=950510c081de4ab6b12f383c0748450d)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:19.689454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0c341-b30e-441f-b802-450d825bff7d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:19.709936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 117/234] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:26:19 2022] PUT /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e/users/4c5a1f97dcc74239afdf377346a19a5a/roles/950510c081de4ab6b12f383c0748450d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:20.435637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4612f-1de8-4dbe-91f3-881f31114808 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:20.436204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4612f-1de8-4dbe-91f3-881f31114808 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:20.436204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4612f-1de8-4dbe-91f3-881f31114808 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:20.437707 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 118/235] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:20.443147 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44883a69-eeff-41ee-9a02-d993a4235497 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:20.443555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44883a69-eeff-41ee-9a02-d993a4235497 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:20.443748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44883a69-eeff-41ee-9a02-d993a4235497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:20.447180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f16eb-2ff9-4103-8f15-f4c1cf9d0b8c None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:20.447464 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f16eb-2ff9-4103-8f15-f4c1cf9d0b8c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:20.447740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f16eb-2ff9-4103-8f15-f4c1cf9d0b8c None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:20.449406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 119/236] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:26:20.453783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f0ed6-cb88-4acd-a6d2-532f14d44b0c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:20.454101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f0ed6-cb88-4acd-a6d2-532f14d44b0c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:20.454379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f0ed6-cb88-4acd-a6d2-532f14d44b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:20.483530 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f18f0ed6-cb88-4acd-a6d2-532f14d44b0c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:20.484768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f18f0ed6-cb88-4acd-a6d2-532f14d44b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:20.487992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-44883a69-eeff-41ee-9a02-d993a4235497 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:20.489211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-44883a69-eeff-41ee-9a02-d993a4235497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:20.549331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:20.607500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 118/237] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:20.609637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 120/238] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:20.634388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44883a69-eeff-41ee-9a02-d993a4235497 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:20.642849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:20.645737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f0ed6-cb88-4acd-a6d2-532f14d44b0c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:20.814188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44883a69-eeff-41ee-9a02-d993a4235497 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:20.833527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f0ed6-cb88-4acd-a6d2-532f14d44b0c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:20.861604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f7184-654c-4771-bfbc-1272b5f5d907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:20.863932 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f7184-654c-4771-bfbc-1272b5f5d907 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:20.865306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f7184-654c-4771-bfbc-1272b5f5d907 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:20.865786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f7184-654c-4771-bfbc-1272b5f5d907 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:20.870109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f7184-654c-4771-bfbc-1272b5f5d907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:20.872284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f7184-654c-4771-bfbc-1272b5f5d907 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:20.875626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 119/239] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:20.891311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f7184-654c-4771-bfbc-1272b5f5d907 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:20.905833 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dbdc5-a09b-4fd2-9686-60e09dc847dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:20.906984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dbdc5-a09b-4fd2-9686-60e09dc847dc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:20.907520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dbdc5-a09b-4fd2-9686-60e09dc847dc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:20.908012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dbdc5-a09b-4fd2-9686-60e09dc847dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:20.928483 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3dbdc5-a09b-4fd2-9686-60e09dc847dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-fd3dbdc5-a09b-4fd2-9686-60e09dc847dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:20.941345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:20.942259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:20.943026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:20.943026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:20.943026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:20.943026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:20.943026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:20.943026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:20.943026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:20.943026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:20.943026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:20.943026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:20.943026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:20.943026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:20.949445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 121/240] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:20.955305 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dbdc5-a09b-4fd2-9686-60e09dc847dc None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:21.014265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:21.017519 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:21.054965 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f7184-654c-4771-bfbc-1272b5f5d907 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:21.092211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39c2b5-4b1c-4787-9f6d-03cb51ec78c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:21.093218 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39c2b5-4b1c-4787-9f6d-03cb51ec78c6 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:21.093715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39c2b5-4b1c-4787-9f6d-03cb51ec78c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:21.094361 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39c2b5-4b1c-4787-9f6d-03cb51ec78c6 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:21.095513 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39c2b5-4b1c-4787-9f6d-03cb51ec78c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:21.097149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39c2b5-4b1c-4787-9f6d-03cb51ec78c6 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:21.134072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 120/241] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:26:20 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:21.171742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dbdc5-a09b-4fd2-9686-60e09dc847dc None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:21.217406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f56714-9894-4939-b215-45a1e58e9034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:21.218575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f56714-9894-4939-b215-45a1e58e9034 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:21.219060 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f56714-9894-4939-b215-45a1e58e9034 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:21.219556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f56714-9894-4939-b215-45a1e58e9034 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:21.220515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f56714-9894-4939-b215-45a1e58e9034 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:21.221020 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f56714-9894-4939-b215-45a1e58e9034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:21.222667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f56714-9894-4939-b215-45a1e58e9034 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:21.233263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 122/242] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:21.242298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39c2b5-4b1c-4787-9f6d-03cb51ec78c6 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:21.390131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39c2b5-4b1c-4787-9f6d-03cb51ec78c6 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:21.420935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:21.427684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d02d27-d5df-4b87-8870-a77c1608b840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:21.428767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d02d27-d5df-4b87-8870-a77c1608b840 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:21.429234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d02d27-d5df-4b87-8870-a77c1608b840 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:21.429867 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d02d27-d5df-4b87-8870-a77c1608b840 None admin] PATH_INFO: `/v3/users/9ea5f0569ca74098b15427ab682fffde` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:21.445596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d02d27-d5df-4b87-8870-a77c1608b840 None admin] RBAC: Authorizing `identity:get_user(user_id=9ea5f0569ca74098b15427ab682fffde)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:21.447107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d02d27-d5df-4b87-8870-a77c1608b840 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:21.447836 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb20e45-eca3-4253-9b08-5a37a3287fbb None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:21.448205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb20e45-eca3-4253-9b08-5a37a3287fbb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:21.448424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb20e45-eca3-4253-9b08-5a37a3287fbb None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:21.449092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 121/243] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:21 2022] GET /identity/v3/users/9ea5f0569ca74098b15427ab682fffde => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:21.449816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 123/244] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:21.454436 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d02d27-d5df-4b87-8870-a77c1608b840 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:21.455263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34ddcb-af06-4bf3-a139-71bd159fc9d7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:21.455606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34ddcb-af06-4bf3-a139-71bd159fc9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:21.455891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34ddcb-af06-4bf3-a139-71bd159fc9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:21.484890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2f34ddcb-af06-4bf3-a139-71bd159fc9d7 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:21.486744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2f34ddcb-af06-4bf3-a139-71bd159fc9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:21.577786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 124/245] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:21.592414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d02d27-d5df-4b87-8870-a77c1608b840 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:21.604083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34ddcb-af06-4bf3-a139-71bd159fc9d7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:21.631748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd13bde-db8c-4b39-bc61-22c8d95129e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:21.632452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd13bde-db8c-4b39-bc61-22c8d95129e4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:21.632737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd13bde-db8c-4b39-bc61-22c8d95129e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:21.634823 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd13bde-db8c-4b39-bc61-22c8d95129e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:21.650331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:21.721405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:21.738573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd13bde-db8c-4b39-bc61-22c8d95129e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:21.740889 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd13bde-db8c-4b39-bc61-22c8d95129e4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:21.753489 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34ddcb-af06-4bf3-a139-71bd159fc9d7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:21.799147 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2a23a-a170-44d4-b7ec-f61ed43051aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:21.799770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2a23a-a170-44d4-b7ec-f61ed43051aa None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:21.800097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2a23a-a170-44d4-b7ec-f61ed43051aa None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:21.800375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2a23a-a170-44d4-b7ec-f61ed43051aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:21.812400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2a23a-a170-44d4-b7ec-f61ed43051aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a2c2a23a-a170-44d4-b7ec-f61ed43051aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:21.821342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:21.822265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:21.824094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 125/246] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:26:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:26:21.824964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438a647-f8e6-4bfc-9738-5138fef17df6 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:21.824964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438a647-f8e6-4bfc-9738-5138fef17df6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:21.824964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438a647-f8e6-4bfc-9738-5138fef17df6 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:21.826467 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 126/247] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:21.831015 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438a647-f8e6-4bfc-9738-5138fef17df6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:21.885478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 122/248] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:21.888629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7f5d2-36b5-471b-a6bf-e6beb6322188 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:21.889124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7f5d2-36b5-471b-a6bf-e6beb6322188 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:21.889858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7f5d2-36b5-471b-a6bf-e6beb6322188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:21.921128 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-08a7f5d2-36b5-471b-a6bf-e6beb6322188 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:21.923350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-08a7f5d2-36b5-471b-a6bf-e6beb6322188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:21.943356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:21.967225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438a647-f8e6-4bfc-9738-5138fef17df6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:22.007016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214ed4f-5919-4834-854c-7ac607a65cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:22.008033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214ed4f-5919-4834-854c-7ac607a65cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:22.008537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214ed4f-5919-4834-854c-7ac607a65cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:22.009010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214ed4f-5919-4834-854c-7ac607a65cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:22.009997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214ed4f-5919-4834-854c-7ac607a65cb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:22.011538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214ed4f-5919-4834-854c-7ac607a65cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:22.013140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214ed4f-5919-4834-854c-7ac607a65cb0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:22.023161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 127/249] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:22.024412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214ed4f-5919-4834-854c-7ac607a65cb0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:22.075326 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 123/250] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:22.076232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7f5d2-36b5-471b-a6bf-e6beb6322188 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:22.100607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:22.145393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214ed4f-5919-4834-854c-7ac607a65cb0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:22.206821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29358250-c16a-470b-8e54-c7dcf5c568e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:22.207688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29358250-c16a-470b-8e54-c7dcf5c568e4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:22.207966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29358250-c16a-470b-8e54-c7dcf5c568e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:22.208238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29358250-c16a-470b-8e54-c7dcf5c568e4 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:22.209614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29358250-c16a-470b-8e54-c7dcf5c568e4 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:22.211106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29358250-c16a-470b-8e54-c7dcf5c568e4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:22.212812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 128/251] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:22 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:22.213565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29358250-c16a-470b-8e54-c7dcf5c568e4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:22.250075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:22.269302 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7f5d2-36b5-471b-a6bf-e6beb6322188 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:22.317082 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee00e0-b785-408a-b61f-eea16382d033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:22.317989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee00e0-b785-408a-b61f-eea16382d033 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:22.318272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee00e0-b785-408a-b61f-eea16382d033 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:22.318548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee00e0-b785-408a-b61f-eea16382d033 None admin] PATH_INFO: `/v3/users/4c5a1f97dcc74239afdf377346a19a5a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:22.319954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee00e0-b785-408a-b61f-eea16382d033 None admin] RBAC: Authorizing `identity:get_user(user_id=4c5a1f97dcc74239afdf377346a19a5a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:22.321406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee00e0-b785-408a-b61f-eea16382d033 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:22.326074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 124/252] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:22 2022] GET /identity/v3/users/4c5a1f97dcc74239afdf377346a19a5a => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:22.327171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee00e0-b785-408a-b61f-eea16382d033 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:22.360458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29358250-c16a-470b-8e54-c7dcf5c568e4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:22.397883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a4750-100f-49c4-959f-575378ebcc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:22.399052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a4750-100f-49c4-959f-575378ebcc8c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:22.399579 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a4750-100f-49c4-959f-575378ebcc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:22.399913 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a4750-100f-49c4-959f-575378ebcc8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:22.413806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a4750-100f-49c4-959f-575378ebcc8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:22.420794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-670a4750-100f-49c4-959f-575378ebcc8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:22.425258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:22.426170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:22.426938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:22.426938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:22.426938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:22.426938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:22.426938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:22.426938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:22.426938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:22.426938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:22.426938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:22.426938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:22.426938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:22.426938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:22.429997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 129/253] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:22.431308 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215c000-20f2-4d47-8902-81f81ddf3821 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:22.431592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215c000-20f2-4d47-8902-81f81ddf3821 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:22.431866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215c000-20f2-4d47-8902-81f81ddf3821 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:22.434686 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 130/254] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:22.435421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215c000-20f2-4d47-8902-81f81ddf3821 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:22.447347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee00e0-b785-408a-b61f-eea16382d033 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:22.488438 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da768d4-7bb5-45a6-ab0a-094a42621810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:22.489158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da768d4-7bb5-45a6-ab0a-094a42621810 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:22.489440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da768d4-7bb5-45a6-ab0a-094a42621810 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:22.489715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da768d4-7bb5-45a6-ab0a-094a42621810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:22.490638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da768d4-7bb5-45a6-ab0a-094a42621810 None admin] RBAC: Adding query filter params (role.id=06b987ffe3ed42d1bc1963d1459eaaa6, scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e, user.id=9ea5f0569ca74098b15427ab682fffde) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:22.490953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da768d4-7bb5-45a6-ab0a-094a42621810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:22.492411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da768d4-7bb5-45a6-ab0a-094a42621810 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:22.499054 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 125/255] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:22 2022] GET /identity/v3/role_assignments?user.id=9ea5f0569ca74098b15427ab682fffde&scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e&role.id=06b987ffe3ed42d1bc1963d1459eaaa6 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:22.499838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da768d4-7bb5-45a6-ab0a-094a42621810 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:22.532118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:22.575488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215c000-20f2-4d47-8902-81f81ddf3821 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:22.618782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d67aa-13d3-4470-a13c-4661daca7076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:22.619452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d67aa-13d3-4470-a13c-4661daca7076 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:22.619737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d67aa-13d3-4470-a13c-4661daca7076 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:22.620007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d67aa-13d3-4470-a13c-4661daca7076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:22.706711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d67aa-13d3-4470-a13c-4661daca7076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:22.708578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d67aa-13d3-4470-a13c-4661daca7076 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:22.744099 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da768d4-7bb5-45a6-ab0a-094a42621810 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:22.795716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:22.802331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc363da6-7601-4058-8f45-9f6cbad4141b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:22.803019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc363da6-7601-4058-8f45-9f6cbad4141b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:22.803306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc363da6-7601-4058-8f45-9f6cbad4141b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:22.803584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc363da6-7601-4058-8f45-9f6cbad4141b None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:22.804456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc363da6-7601-4058-8f45-9f6cbad4141b None admin] RBAC: Adding query filter params (name=member) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:22.804768 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc363da6-7601-4058-8f45-9f6cbad4141b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:22.806334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc363da6-7601-4058-8f45-9f6cbad4141b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:22.816004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 126/256] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:22.817386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd37f56-0e61-450c-bd9d-d8d5e1d5d325 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:22.817673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd37f56-0e61-450c-bd9d-d8d5e1d5d325 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:22.818843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd37f56-0e61-450c-bd9d-d8d5e1d5d325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:22.836222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 131/257] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:22.837085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d67aa-13d3-4470-a13c-4661daca7076 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:22.843299 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-9cd37f56-0e61-450c-bd9d-d8d5e1d5d325 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:22.844594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9cd37f56-0e61-450c-bd9d-d8d5e1d5d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:22.856340 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:22.958003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d67aa-13d3-4470-a13c-4661daca7076 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:22.961970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 127/258] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:22.962618 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd37f56-0e61-450c-bd9d-d8d5e1d5d325 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:22.996109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94885ed8-8f8f-47a5-9a0f-681fabee50d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:22.996988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94885ed8-8f8f-47a5-9a0f-681fabee50d4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:22.997268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94885ed8-8f8f-47a5-9a0f-681fabee50d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:22.997539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94885ed8-8f8f-47a5-9a0f-681fabee50d4 None admin] PATH_INFO: `/v3/users/3b027f09d3274d9ea581af6d66e8afae` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:23.002763 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94885ed8-8f8f-47a5-9a0f-681fabee50d4 None admin] RBAC: Authorizing `identity:get_user(user_id=3b027f09d3274d9ea581af6d66e8afae)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:23.004239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94885ed8-8f8f-47a5-9a0f-681fabee50d4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:23.007886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 132/259] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:22 2022] GET /identity/v3/users/3b027f09d3274d9ea581af6d66e8afae => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:23.013382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94885ed8-8f8f-47a5-9a0f-681fabee50d4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:23.063181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:23.123982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94885ed8-8f8f-47a5-9a0f-681fabee50d4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:23.159824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d2e03-307d-49f2-80c5-c89f24115138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:23.160938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d2e03-307d-49f2-80c5-c89f24115138 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:23.161333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d2e03-307d-49f2-80c5-c89f24115138 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:23.161677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d2e03-307d-49f2-80c5-c89f24115138 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:23.162984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d2e03-307d-49f2-80c5-c89f24115138 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:23.164518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d2e03-307d-49f2-80c5-c89f24115138 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:23.166637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 133/260] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:23 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:23.169529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d2e03-307d-49f2-80c5-c89f24115138 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:23.191300 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd37f56-0e61-450c-bd9d-d8d5e1d5d325 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:23.261622 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf999f7-b3b8-4011-a857-3f67d2576dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:23.262445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf999f7-b3b8-4011-a857-3f67d2576dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:23.262730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf999f7-b3b8-4011-a857-3f67d2576dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:23.263000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf999f7-b3b8-4011-a857-3f67d2576dc9 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:23.266632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf999f7-b3b8-4011-a857-3f67d2576dc9 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:23.268108 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf999f7-b3b8-4011-a857-3f67d2576dc9 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:23.270066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 128/261] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:22 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:23.270587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf999f7-b3b8-4011-a857-3f67d2576dc9 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:23.279168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d2e03-307d-49f2-80c5-c89f24115138 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:23.290253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:23.296492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:23.316897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7bbbb-7e70-4ae6-af44-0588ab202731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:23.317820 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7bbbb-7e70-4ae6-af44-0588ab202731 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:23.318210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7bbbb-7e70-4ae6-af44-0588ab202731 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:23.318597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7bbbb-7e70-4ae6-af44-0588ab202731 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:23.330982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb7bbbb-7e70-4ae6-af44-0588ab202731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-bbb7bbbb-7e70-4ae6-af44-0588ab202731 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:23.347339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:23.348347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 134/262] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:26:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:23.349598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7bbbb-7e70-4ae6-af44-0588ab202731 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:23.481078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7bbbb-7e70-4ae6-af44-0588ab202731 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:23.527305 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cdab3-c8bc-4251-ac47-f3244dc24737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:23.527899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cdab3-c8bc-4251-ac47-f3244dc24737 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:23.528185 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cdab3-c8bc-4251-ac47-f3244dc24737 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:23.528462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cdab3-c8bc-4251-ac47-f3244dc24737 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:23.529303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cdab3-c8bc-4251-ac47-f3244dc24737 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e, user.id=4c5a1f97dcc74239afdf377346a19a5a) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:23.529613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cdab3-c8bc-4251-ac47-f3244dc24737 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:23.530653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf999f7-b3b8-4011-a857-3f67d2576dc9 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:23.531109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cdab3-c8bc-4251-ac47-f3244dc24737 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:23.552097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 135/263] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:23 2022] GET /identity/v3/role_assignments?user.id=4c5a1f97dcc74239afdf377346a19a5a&scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:23.552745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cdab3-c8bc-4251-ac47-f3244dc24737 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:23.583121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec58178-b901-4af0-a8ba-c0db83c00275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:23.584731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec58178-b901-4af0-a8ba-c0db83c00275 None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:23.585165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec58178-b901-4af0-a8ba-c0db83c00275 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:23.585589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec58178-b901-4af0-a8ba-c0db83c00275 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e/users/3b027f09d3274d9ea581af6d66e8afae/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:23.590919 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec58178-b901-4af0-a8ba-c0db83c00275 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb4447b59c0746648ea2c2d04f5e3c7e, user_id=3b027f09d3274d9ea581af6d66e8afae, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:23.592603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec58178-b901-4af0-a8ba-c0db83c00275 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:23.618090 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 129/264] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:26:23 2022] PUT /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e/users/3b027f09d3274d9ea581af6d66e8afae/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:23.645191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:23.732448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cdab3-c8bc-4251-ac47-f3244dc24737 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:23.774146 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406b573-0b34-4935-8731-cd3c3ecff2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:23.774841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406b573-0b34-4935-8731-cd3c3ecff2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:23.775131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406b573-0b34-4935-8731-cd3c3ecff2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:23.775409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406b573-0b34-4935-8731-cd3c3ecff2e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:23.776180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406b573-0b34-4935-8731-cd3c3ecff2e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:23.776525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406b573-0b34-4935-8731-cd3c3ecff2e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:23.778050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406b573-0b34-4935-8731-cd3c3ecff2e9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:23.787966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 136/265] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:23.793661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec58178-b901-4af0-a8ba-c0db83c00275 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:23.873501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0566-0269-49f4-8056-e9d7d6c6a593 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:23.874047 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0566-0269-49f4-8056-e9d7d6c6a593 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:23.874461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0566-0269-49f4-8056-e9d7d6c6a593 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:23.875993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 137/266] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:26:23.881310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbf11e-848b-4419-aec0-557853bcab59 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:23.881785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbf11e-848b-4419-aec0-557853bcab59 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:23.881899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:23.882364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbf11e-848b-4419-aec0-557853bcab59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:23.906926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e3bbf11e-848b-4419-aec0-557853bcab59 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:23.908433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e3bbf11e-848b-4419-aec0-557853bcab59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:23.931934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec58178-b901-4af0-a8ba-c0db83c00275 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:23.970880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca4c75-7bd3-414e-a9f7-0d8363afcbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:23.971764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca4c75-7bd3-414e-a9f7-0d8363afcbdd None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:23.972047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca4c75-7bd3-414e-a9f7-0d8363afcbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:23.972323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca4c75-7bd3-414e-a9f7-0d8363afcbdd None admin] PATH_INFO: `/v3/users/4c5a1f97dcc74239afdf377346a19a5a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:23.973521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca4c75-7bd3-414e-a9f7-0d8363afcbdd None admin] RBAC: Authorizing `identity:get_user(user_id=4c5a1f97dcc74239afdf377346a19a5a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:23.975051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca4c75-7bd3-414e-a9f7-0d8363afcbdd None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:23.977051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 130/267] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:23 2022] GET /identity/v3/users/4c5a1f97dcc74239afdf377346a19a5a => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:23.982945 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca4c75-7bd3-414e-a9f7-0d8363afcbdd None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:24.001939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 138/268] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:24.021764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbf11e-848b-4419-aec0-557853bcab59 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:24.062279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:24.096532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca4c75-7bd3-414e-a9f7-0d8363afcbdd None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:24.133081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e97e8-cd0f-4494-b952-ae91356e729d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:24.133736 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e97e8-cd0f-4494-b952-ae91356e729d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:24.134067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e97e8-cd0f-4494-b952-ae91356e729d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:24.134346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e97e8-cd0f-4494-b952-ae91356e729d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:24.143389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:24.184207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbf11e-848b-4419-aec0-557853bcab59 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:24.210817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e97e8-cd0f-4494-b952-ae91356e729d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:24.212508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e97e8-cd0f-4494-b952-ae91356e729d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:24.229130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057a61d-cb8e-4503-8683-3be64b79f166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:24.230760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057a61d-cb8e-4503-8683-3be64b79f166 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:24.231051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057a61d-cb8e-4503-8683-3be64b79f166 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:24.231322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057a61d-cb8e-4503-8683-3be64b79f166 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:24.242058 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057a61d-cb8e-4503-8683-3be64b79f166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-2057a61d-cb8e-4503-8683-3be64b79f166 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:24.251635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:24.252536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:24.253323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 139/269] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:26:24.253992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf8835-6db1-4552-8778-33d7768a92a9 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:24.254816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf8835-6db1-4552-8778-33d7768a92a9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:24.254816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf8835-6db1-4552-8778-33d7768a92a9 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:24.255893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 140/270] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:24.260638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf8835-6db1-4552-8778-33d7768a92a9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:24.332743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 131/271] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:26:24.334226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a0d7a-68db-49dd-860b-833c9e4b8d3a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:24.334587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a0d7a-68db-49dd-860b-833c9e4b8d3a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:24.334936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a0d7a-68db-49dd-860b-833c9e4b8d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:24.352156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:24.356598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-6f5a0d7a-68db-49dd-860b-833c9e4b8d3a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:24.358848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6f5a0d7a-68db-49dd-860b-833c9e4b8d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:24.372444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf8835-6db1-4552-8778-33d7768a92a9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:24.406277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:24.415399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0789f1-c66e-4545-9d23-18204398ca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:24.416060 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0789f1-c66e-4545-9d23-18204398ca5c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:24.416367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0789f1-c66e-4545-9d23-18204398ca5c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:24.416605 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0789f1-c66e-4545-9d23-18204398ca5c None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:24.418198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0789f1-c66e-4545-9d23-18204398ca5c None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:24.418524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0789f1-c66e-4545-9d23-18204398ca5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:24.419955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0789f1-c66e-4545-9d23-18204398ca5c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:24.428923 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 141/272] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:24.430363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0789f1-c66e-4545-9d23-18204398ca5c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:24.441656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 132/273] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:24.443166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a0d7a-68db-49dd-860b-833c9e4b8d3a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:24.539054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0789f1-c66e-4545-9d23-18204398ca5c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:24.570594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a0d7a-68db-49dd-860b-833c9e4b8d3a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:24.577351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089ee90-ad91-45ca-b40b-2590be060ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:24.578241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089ee90-ad91-45ca-b40b-2590be060ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:24.578523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089ee90-ad91-45ca-b40b-2590be060ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:24.578795 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089ee90-ad91-45ca-b40b-2590be060ff0 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:24.579990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089ee90-ad91-45ca-b40b-2590be060ff0 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:24.581710 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089ee90-ad91-45ca-b40b-2590be060ff0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:24.583528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 142/274] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:24 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:24.584299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089ee90-ad91-45ca-b40b-2590be060ff0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:24.600428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:24.617350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a458ed-9ef7-48ac-a8b6-f323b6787376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:24.618948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a458ed-9ef7-48ac-a8b6-f323b6787376 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:24.619655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a458ed-9ef7-48ac-a8b6-f323b6787376 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:24.620105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a458ed-9ef7-48ac-a8b6-f323b6787376 None admin] PATH_INFO: `/v3/users/23ea533b79dc4c88adce6095cbb8d950` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:24.638042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a458ed-9ef7-48ac-a8b6-f323b6787376 None admin] RBAC: Authorizing `identity:get_user(user_id=23ea533b79dc4c88adce6095cbb8d950)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:24.640085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a458ed-9ef7-48ac-a8b6-f323b6787376 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:24.642625 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 133/275] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:24 2022] GET /identity/v3/users/23ea533b79dc4c88adce6095cbb8d950 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:24.643815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a458ed-9ef7-48ac-a8b6-f323b6787376 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:24.673290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:24.710490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089ee90-ad91-45ca-b40b-2590be060ff0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:24.750486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a458ed-9ef7-48ac-a8b6-f323b6787376 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:24.757343 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027394eb-8fd6-47b9-bb0e-42fbdb0cf7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:24.758351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027394eb-8fd6-47b9-bb0e-42fbdb0cf7cf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:24.758782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027394eb-8fd6-47b9-bb0e-42fbdb0cf7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:24.759200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027394eb-8fd6-47b9-bb0e-42fbdb0cf7cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:24.761071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027394eb-8fd6-47b9-bb0e-42fbdb0cf7cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:24.763266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027394eb-8fd6-47b9-bb0e-42fbdb0cf7cf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:24.765773 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 143/276] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:24.767052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027394eb-8fd6-47b9-bb0e-42fbdb0cf7cf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:24.788591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bea9f7-3d36-4979-b6af-091855725940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:24.789437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bea9f7-3d36-4979-b6af-091855725940 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:24.789824 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bea9f7-3d36-4979-b6af-091855725940 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:24.790235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bea9f7-3d36-4979-b6af-091855725940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:24.791225 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bea9f7-3d36-4979-b6af-091855725940 None admin] RBAC: Adding query filter params (role.id=950510c081de4ab6b12f383c0748450d, scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e, user.id=4c5a1f97dcc74239afdf377346a19a5a) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:24.791633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bea9f7-3d36-4979-b6af-091855725940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:24.793773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bea9f7-3d36-4979-b6af-091855725940 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:24.800605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 134/277] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:24 2022] GET /identity/v3/role_assignments?user.id=4c5a1f97dcc74239afdf377346a19a5a&scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e&role.id=950510c081de4ab6b12f383c0748450d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:24.801586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bea9f7-3d36-4979-b6af-091855725940 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:24.890647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:24.903630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:24.919521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027394eb-8fd6-47b9-bb0e-42fbdb0cf7cf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:24.923928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bea9f7-3d36-4979-b6af-091855725940 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:24.958445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf2b8c-689a-4610-9b6b-6a9e6e170ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:24.959328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf2b8c-689a-4610-9b6b-6a9e6e170ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:24.959938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf2b8c-689a-4610-9b6b-6a9e6e170ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:24.960055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf2b8c-689a-4610-9b6b-6a9e6e170ba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:24.960995 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf2b8c-689a-4610-9b6b-6a9e6e170ba8 None admin] RBAC: Adding query filter params (role.id=86e07561a0854fc28db2e9f357e53d66, user.id=23ea533b79dc4c88adce6095cbb8d950, scope.system=all) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:24.961409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf2b8c-689a-4610-9b6b-6a9e6e170ba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:24.963064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf2b8c-689a-4610-9b6b-6a9e6e170ba8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:24.970328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 144/278] 10.222.0.60 () {56 vars in 1308 bytes} [Mon Jul 25 12:26:24 2022] GET /identity/v3/role_assignments?user.id=23ea533b79dc4c88adce6095cbb8d950&scope.system=all&role.id=86e07561a0854fc28db2e9f357e53d66 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:24.983641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdac333-aa6e-4f34-a89a-008511c2c532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:24.984734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdac333-aa6e-4f34-a89a-008511c2c532 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:24.985223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdac333-aa6e-4f34-a89a-008511c2c532 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:24.985691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdac333-aa6e-4f34-a89a-008511c2c532 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:24.986819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdac333-aa6e-4f34-a89a-008511c2c532 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:24.988585 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdac333-aa6e-4f34-a89a-008511c2c532 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:25.037261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 135/279] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:26:24 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:25.474728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e49f7-5626-4301-8121-d9c4eeca4156 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:25.475065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e49f7-5626-4301-8121-d9c4eeca4156 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:25.475273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e49f7-5626-4301-8121-d9c4eeca4156 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:25.476790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 145/280] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:25.483096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb678b-4788-41fb-8b14-afe584056212 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:25.483628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb678b-4788-41fb-8b14-afe584056212 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:25.484126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb678b-4788-41fb-8b14-afe584056212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:25.505125 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b1eb678b-4788-41fb-8b14-afe584056212 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:25.506694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b1eb678b-4788-41fb-8b14-afe584056212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:25.592241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 136/281] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:25.631570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e49f7-5626-4301-8121-d9c4eeca4156 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:25.768166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e49f7-5626-4301-8121-d9c4eeca4156 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:25.811183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ccde5-72ce-4cb4-b35b-fe631014f7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:25.811976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ccde5-72ce-4cb4-b35b-fe631014f7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:25.812255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ccde5-72ce-4cb4-b35b-fe631014f7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:25.812517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ccde5-72ce-4cb4-b35b-fe631014f7e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:25.825663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ccde5-72ce-4cb4-b35b-fe631014f7e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b97ccde5-72ce-4cb4-b35b-fe631014f7e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:25.838075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:25.839505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:25.840300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:25.840300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:25.840300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:25.840300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:25.840300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:25.840300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:25.840300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:25.840300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:25.840300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:25.840300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:25.840300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:25.840300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:25.842772 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 146/282] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:25.848766 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb678b-4788-41fb-8b14-afe584056212 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:25.958341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:25.994544 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb678b-4788-41fb-8b14-afe584056212 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:26.040149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c36bf-8ad0-4cf1-af3f-b1fa259fdd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:26.041175 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c36bf-8ad0-4cf1-af3f-b1fa259fdd7f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:26.041731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c36bf-8ad0-4cf1-af3f-b1fa259fdd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:26.042794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c36bf-8ad0-4cf1-af3f-b1fa259fdd7f None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:26.043760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c36bf-8ad0-4cf1-af3f-b1fa259fdd7f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:26.044336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c36bf-8ad0-4cf1-af3f-b1fa259fdd7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:26.046004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c36bf-8ad0-4cf1-af3f-b1fa259fdd7f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:26.048879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31daa769-e5db-48b2-ba2a-8e0ad2721c02 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:26.049385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31daa769-e5db-48b2-ba2a-8e0ad2721c02 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:26.049893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31daa769-e5db-48b2-ba2a-8e0ad2721c02 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:26.051509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 147/283] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:26.058838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 137/284] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:26.064294 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31daa769-e5db-48b2-ba2a-8e0ad2721c02 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:26.067456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dd433-19c6-4fb8-9bd9-40c8ff4c946e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:26.067746 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dd433-19c6-4fb8-9bd9-40c8ff4c946e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:26.068028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dd433-19c6-4fb8-9bd9-40c8ff4c946e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:26.078310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:26.107165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-9d0dd433-19c6-4fb8-9bd9-40c8ff4c946e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:26.108372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9d0dd433-19c6-4fb8-9bd9-40c8ff4c946e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:26.214578 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 138/285] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:26.231001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0dd433-19c6-4fb8-9bd9-40c8ff4c946e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:26.298738 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31daa769-e5db-48b2-ba2a-8e0ad2721c02 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:26.299905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:26.335871 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0dd433-19c6-4fb8-9bd9-40c8ff4c946e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:26.373680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c216d-9da0-4853-9867-ef38137a9f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:26.374597 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c216d-9da0-4853-9867-ef38137a9f74 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:26.374891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c216d-9da0-4853-9867-ef38137a9f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:26.375169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c216d-9da0-4853-9867-ef38137a9f74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:26.379933 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126dface-6df2-44f6-b372-e88a73508e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:26.380811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126dface-6df2-44f6-b372-e88a73508e17 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:26.381096 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126dface-6df2-44f6-b372-e88a73508e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:26.381379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126dface-6df2-44f6-b372-e88a73508e17 None admin] PATH_INFO: `/v3/users/9ea5f0569ca74098b15427ab682fffde` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:26.386612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126dface-6df2-44f6-b372-e88a73508e17 None admin] RBAC: Authorizing `identity:get_user(user_id=9ea5f0569ca74098b15427ab682fffde)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:26.388078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126dface-6df2-44f6-b372-e88a73508e17 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:26.389748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 148/286] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:26 2022] GET /identity/v3/users/9ea5f0569ca74098b15427ab682fffde => generated 297 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:26.395523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126dface-6df2-44f6-b372-e88a73508e17 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:26.396361 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c216d-9da0-4853-9867-ef38137a9f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-2b7c216d-9da0-4853-9867-ef38137a9f74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:26.404765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:26.405628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:26.406479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:26.406479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:26.406479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:26.406479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:26.406479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:26.406479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:26.406479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:26.406479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:26.406479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:26.406479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:26.406479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:26.406479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:26.413707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 139/287] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:26.419545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c216d-9da0-4853-9867-ef38137a9f74 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:26.530289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:26.531285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c216d-9da0-4853-9867-ef38137a9f74 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:26.558736 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:26.569307 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126dface-6df2-44f6-b372-e88a73508e17 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:26.571856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09874fee-0b76-4341-9f59-8f6719af7f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:26.572574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09874fee-0b76-4341-9f59-8f6719af7f14 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:26.572858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09874fee-0b76-4341-9f59-8f6719af7f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:26.573137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09874fee-0b76-4341-9f59-8f6719af7f14 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:26.573988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09874fee-0b76-4341-9f59-8f6719af7f14 None admin] RBAC: Adding query filter params (name=member) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:26.574305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09874fee-0b76-4341-9f59-8f6719af7f14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:26.575737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09874fee-0b76-4341-9f59-8f6719af7f14 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:26.599310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 140/288] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:26.602470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09874fee-0b76-4341-9f59-8f6719af7f14 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:26.639165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8682a-9c00-40f8-87c4-e199048159f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:26.640035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8682a-9c00-40f8-87c4-e199048159f2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:26.640314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8682a-9c00-40f8-87c4-e199048159f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:26.640590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8682a-9c00-40f8-87c4-e199048159f2 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:26.645174 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8682a-9c00-40f8-87c4-e199048159f2 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:26.646655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8682a-9c00-40f8-87c4-e199048159f2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:26.648397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 149/289] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:26 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:26.655679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8682a-9c00-40f8-87c4-e199048159f2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:26.751661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09874fee-0b76-4341-9f59-8f6719af7f14 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:26.789304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c657e02-b5e0-4c51-9caa-f46058ccab98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:26.790219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c657e02-b5e0-4c51-9caa-f46058ccab98 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:26.790658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c657e02-b5e0-4c51-9caa-f46058ccab98 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:26.790939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c657e02-b5e0-4c51-9caa-f46058ccab98 None admin] PATH_INFO: `/v3/users/3b027f09d3274d9ea581af6d66e8afae` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:26.792705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c657e02-b5e0-4c51-9caa-f46058ccab98 None admin] RBAC: Authorizing `identity:get_user(user_id=3b027f09d3274d9ea581af6d66e8afae)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:26.794198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c657e02-b5e0-4c51-9caa-f46058ccab98 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:26.796997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 141/290] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:26 2022] GET /identity/v3/users/3b027f09d3274d9ea581af6d66e8afae => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:26.804011 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c657e02-b5e0-4c51-9caa-f46058ccab98 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:26.876811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:26.884102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8682a-9c00-40f8-87c4-e199048159f2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:26.923339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c657e02-b5e0-4c51-9caa-f46058ccab98 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:26.962143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0122f-8044-4b31-9f4e-1bb6e4630c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:26.962801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0122f-8044-4b31-9f4e-1bb6e4630c6c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:26.963087 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0122f-8044-4b31-9f4e-1bb6e4630c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:26.963362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0122f-8044-4b31-9f4e-1bb6e4630c6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:26.965145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9984fd1-0740-48fd-9b71-df23c86883be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:26.966053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9984fd1-0740-48fd-9b71-df23c86883be None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:26.966334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9984fd1-0740-48fd-9b71-df23c86883be None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:26.966608 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9984fd1-0740-48fd-9b71-df23c86883be None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e/users/9ea5f0569ca74098b15427ab682fffde/roles/06b987ffe3ed42d1bc1963d1459eaaa6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:26.971281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9984fd1-0740-48fd-9b71-df23c86883be None admin] RBAC: Authorizing `identity:create_grant(project_id=eb4447b59c0746648ea2c2d04f5e3c7e, user_id=9ea5f0569ca74098b15427ab682fffde, role_id=06b987ffe3ed42d1bc1963d1459eaaa6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:26.972735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9984fd1-0740-48fd-9b71-df23c86883be None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:27.002816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 150/291] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:26:26 2022] PUT /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e/users/9ea5f0569ca74098b15427ab682fffde/roles/06b987ffe3ed42d1bc1963d1459eaaa6 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:27.060014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0122f-8044-4b31-9f4e-1bb6e4630c6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:27.061521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0122f-8044-4b31-9f4e-1bb6e4630c6c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:27.126110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:27.185057 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 142/292] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:27.190686 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9984fd1-0740-48fd-9b71-df23c86883be None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:27.202595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:27.297984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff9aaa-3bd0-4af5-be01-3762c93017ac None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:27.298327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff9aaa-3bd0-4af5-be01-3762c93017ac None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:27.298521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff9aaa-3bd0-4af5-be01-3762c93017ac None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:27.300197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 143/293] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:27.305474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9984fd1-0740-48fd-9b71-df23c86883be None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:27.306039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03b654-c2ca-4258-ad02-72ee3feb2304 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:27.306556 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03b654-c2ca-4258-ad02-72ee3feb2304 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:27.307035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03b654-c2ca-4258-ad02-72ee3feb2304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:27.327980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-aa03b654-c2ca-4258-ad02-72ee3feb2304 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:27.329455 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-aa03b654-c2ca-4258-ad02-72ee3feb2304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:27.350090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc95af8-7c1a-4e97-a602-424b97adb850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:27.350996 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc95af8-7c1a-4e97-a602-424b97adb850 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:27.351281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc95af8-7c1a-4e97-a602-424b97adb850 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:27.351574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc95af8-7c1a-4e97-a602-424b97adb850 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:27.353568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc95af8-7c1a-4e97-a602-424b97adb850 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:27.355925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc95af8-7c1a-4e97-a602-424b97adb850 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:27.358326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 151/294] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:27 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:27.366042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc95af8-7c1a-4e97-a602-424b97adb850 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:27.431956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 144/295] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:27.460907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03b654-c2ca-4258-ad02-72ee3feb2304 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:27.475615 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:27.546779 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc95af8-7c1a-4e97-a602-424b97adb850 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:27.567484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:27.586924 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03b654-c2ca-4258-ad02-72ee3feb2304 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:27.618324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b9152-344a-4dd7-af83-311954e91f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:27.619336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b9152-344a-4dd7-af83-311954e91f4e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:27.619827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b9152-344a-4dd7-af83-311954e91f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:27.621216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b9152-344a-4dd7-af83-311954e91f4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:27.622294 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b9152-344a-4dd7-af83-311954e91f4e None admin] RBAC: Adding query filter params (role.id=86e07561a0854fc28db2e9f357e53d66, scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e, user.id=3b027f09d3274d9ea581af6d66e8afae) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:27.624286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b9152-344a-4dd7-af83-311954e91f4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:27.626005 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b9152-344a-4dd7-af83-311954e91f4e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:27.639156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 152/296] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:27 2022] GET /identity/v3/role_assignments?user.id=3b027f09d3274d9ea581af6d66e8afae&scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e&role.id=86e07561a0854fc28db2e9f357e53d66 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:27.663375 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f18f7-09d3-44f4-b5ad-5513c0320846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:27.664923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f18f7-09d3-44f4-b5ad-5513c0320846 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:27.665414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f18f7-09d3-44f4-b5ad-5513c0320846 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:27.665895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f18f7-09d3-44f4-b5ad-5513c0320846 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:27.669506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c788f-de96-41db-9e14-4a44add1cafb None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:27.670484 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c788f-de96-41db-9e14-4a44add1cafb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:27.670960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c788f-de96-41db-9e14-4a44add1cafb None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:27.672632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 153/297] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:26:27.678227 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f03977-51a2-4516-af10-bb8a3817ff1c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:27.678904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f03977-51a2-4516-af10-bb8a3817ff1c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:27.679406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f03977-51a2-4516-af10-bb8a3817ff1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:27.705118 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f18f7-09d3-44f4-b5ad-5513c0320846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:27.715739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e2f03977-51a2-4516-af10-bb8a3817ff1c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:27.717322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e2f03977-51a2-4516-af10-bb8a3817ff1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-6e6f18f7-09d3-44f4-b5ad-5513c0320846 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:27.720483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:27.721401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:27.722243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:27.722243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:27.722243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:27.722243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:27.722243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:27.722243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:27.722243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:27.722243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:27.722243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:27.722243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:27.722243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:27.722243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:27.724998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 145/298] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:27.731737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f18f7-09d3-44f4-b5ad-5513c0320846 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:27.838519 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 154/299] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:27.866443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f03977-51a2-4516-af10-bb8a3817ff1c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:27.893601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f18f7-09d3-44f4-b5ad-5513c0320846 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:27.938120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ae993-5908-497e-a58a-94f25623992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:27.939182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ae993-5908-497e-a58a-94f25623992d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:27.939658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ae993-5908-497e-a58a-94f25623992d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:27.940117 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ae993-5908-497e-a58a-94f25623992d None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:27.940992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ae993-5908-497e-a58a-94f25623992d None admin] RBAC: Adding query filter params (name=member) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:27.941304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ae993-5908-497e-a58a-94f25623992d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:27.942762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ae993-5908-497e-a58a-94f25623992d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:27.958171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 146/300] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:27.959941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe897525-1c73-446e-ae92-a151de9d033b None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:27.960387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe897525-1c73-446e-ae92-a151de9d033b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:27.960803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe897525-1c73-446e-ae92-a151de9d033b None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:27.962278 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:27.962939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 147/301] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:27.963702 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe897525-1c73-446e-ae92-a151de9d033b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:27.979325 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:28.008391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f03977-51a2-4516-af10-bb8a3817ff1c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:28.043944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dce81e-51ab-4813-8e30-ce6b01a618fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:28.045496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dce81e-51ab-4813-8e30-ce6b01a618fa None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:28.045783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dce81e-51ab-4813-8e30-ce6b01a618fa None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:28.046090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dce81e-51ab-4813-8e30-ce6b01a618fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:28.047278 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dce81e-51ab-4813-8e30-ce6b01a618fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:28.048742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dce81e-51ab-4813-8e30-ce6b01a618fa None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:28.051070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 155/302] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:28.051984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58cb4e-378c-43ed-ad31-c8490ee5cd5a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:28.052274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58cb4e-378c-43ed-ad31-c8490ee5cd5a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:28.052555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58cb4e-378c-43ed-ad31-c8490ee5cd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:28.075046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7d58cb4e-378c-43ed-ad31-c8490ee5cd5a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:28.076239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7d58cb4e-378c-43ed-ad31-c8490ee5cd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:28.081293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe897525-1c73-446e-ae92-a151de9d033b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:28.114625 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a8885-0edd-45de-9c46-bf1224ce9bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:28.115404 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a8885-0edd-45de-9c46-bf1224ce9bdd None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:28.115685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a8885-0edd-45de-9c46-bf1224ce9bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:28.115959 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a8885-0edd-45de-9c46-bf1224ce9bdd None admin] PATH_INFO: `/v3/users/23ea533b79dc4c88adce6095cbb8d950` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:28.117144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a8885-0edd-45de-9c46-bf1224ce9bdd None admin] RBAC: Authorizing `identity:get_user(user_id=23ea533b79dc4c88adce6095cbb8d950)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:28.118617 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a8885-0edd-45de-9c46-bf1224ce9bdd None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:28.125474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 148/303] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:27 2022] GET /identity/v3/users/23ea533b79dc4c88adce6095cbb8d950 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:28.125474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a8885-0edd-45de-9c46-bf1224ce9bdd None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:28.153417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 156/304] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:28.154255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58cb4e-378c-43ed-ad31-c8490ee5cd5a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:28.216717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:28.229601 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:28.241200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a8885-0edd-45de-9c46-bf1224ce9bdd None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:28.271875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58cb4e-378c-43ed-ad31-c8490ee5cd5a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:28.278367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c306b5d-8a6d-43cc-b829-37a8bcba039c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:28.279036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c306b5d-8a6d-43cc-b829-37a8bcba039c None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:28.279319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c306b5d-8a6d-43cc-b829-37a8bcba039c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:28.280002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c306b5d-8a6d-43cc-b829-37a8bcba039c None admin] PATH_INFO: `/v3/groups` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:28.280699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c306b5d-8a6d-43cc-b829-37a8bcba039c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:28.282557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c306b5d-8a6d-43cc-b829-37a8bcba039c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:28.298612 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 149/305] 10.222.0.60 () {60 vars in 1134 bytes} [Mon Jul 25 12:26:28 2022] POST /identity/v3/groups => generated 229 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:28.299327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c306b5d-8a6d-43cc-b829-37a8bcba039c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:28.313410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb47a79-4684-49b8-b9ea-5cffa2652f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:28.314364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb47a79-4684-49b8-b9ea-5cffa2652f6c None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:28.314697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb47a79-4684-49b8-b9ea-5cffa2652f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:28.314977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb47a79-4684-49b8-b9ea-5cffa2652f6c None admin] PATH_INFO: `/v3/system/users/23ea533b79dc4c88adce6095cbb8d950/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:28.316791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb47a79-4684-49b8-b9ea-5cffa2652f6c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=23ea533b79dc4c88adce6095cbb8d950, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:28.318305 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb47a79-4684-49b8-b9ea-5cffa2652f6c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:28.331193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 157/306] 10.222.0.60 () {58 vars in 1334 bytes} [Mon Jul 25 12:26:28 2022] PUT /identity/v3/system/users/23ea533b79dc4c88adce6095cbb8d950/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:28.430056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c306b5d-8a6d-43cc-b829-37a8bcba039c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:28.471904 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdbb4f-e07d-4ee2-b979-bae7d5e94be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:28.473155 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdbb4f-e07d-4ee2-b979-bae7d5e94be8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:28.473646 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdbb4f-e07d-4ee2-b979-bae7d5e94be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:28.474185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdbb4f-e07d-4ee2-b979-bae7d5e94be8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:28.484839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:28.488881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdbb4f-e07d-4ee2-b979-bae7d5e94be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-d4cdbb4f-e07d-4ee2-b979-bae7d5e94be8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:28.498457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:28.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:28.500146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:28.500146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:28.500146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:28.500146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:28.500146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:28.500146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:28.500146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:28.500146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:28.500146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:28.500146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:28.500146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:28.500146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:28.502684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 150/307] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:28.510447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb47a79-4684-49b8-b9ea-5cffa2652f6c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:28.617361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb47a79-4684-49b8-b9ea-5cffa2652f6c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:28.644338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:28.656516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f145-3a90-46ff-8b6f-494e83e9a143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:28.657516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f145-3a90-46ff-8b6f-494e83e9a143 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:28.658024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f145-3a90-46ff-8b6f-494e83e9a143 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:28.658492 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f145-3a90-46ff-8b6f-494e83e9a143 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:28.659433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f145-3a90-46ff-8b6f-494e83e9a143 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:28.659949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f145-3a90-46ff-8b6f-494e83e9a143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:28.661581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f145-3a90-46ff-8b6f-494e83e9a143 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:28.671540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 158/308] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:28.677767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdbb4f-e07d-4ee2-b979-bae7d5e94be8 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:28.783570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdbb4f-e07d-4ee2-b979-bae7d5e94be8 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:28.818793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01abc61-a504-4ddd-bb36-43bb783ab17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:28.820013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01abc61-a504-4ddd-bb36-43bb783ab17b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:28.820494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01abc61-a504-4ddd-bb36-43bb783ab17b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:28.820962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01abc61-a504-4ddd-bb36-43bb783ab17b None admin] PATH_INFO: `/v3/users/3441a0a2da854eef9b3847539b2a8cc6` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:28.835053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01abc61-a504-4ddd-bb36-43bb783ab17b None admin] RBAC: Authorizing `identity:get_user(user_id=3441a0a2da854eef9b3847539b2a8cc6)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:28.836808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01abc61-a504-4ddd-bb36-43bb783ab17b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:28.839570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 151/309] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:28 2022] GET /identity/v3/users/3441a0a2da854eef9b3847539b2a8cc6 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:28.845468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f145-3a90-46ff-8b6f-494e83e9a143 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:28.950813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f145-3a90-46ff-8b6f-494e83e9a143 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:28.986769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b703f-4573-4a2d-a9b9-0e96f20e3d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:28.987832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b703f-4573-4a2d-a9b9-0e96f20e3d54 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:28.988354 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b703f-4573-4a2d-a9b9-0e96f20e3d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:28.988818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b703f-4573-4a2d-a9b9-0e96f20e3d54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:28.989839 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b703f-4573-4a2d-a9b9-0e96f20e3d54 None admin] RBAC: Adding query filter params (role.id=06b987ffe3ed42d1bc1963d1459eaaa6, user.id=3441a0a2da854eef9b3847539b2a8cc6, scope.system=all) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:28.990415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b703f-4573-4a2d-a9b9-0e96f20e3d54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:28.992121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b703f-4573-4a2d-a9b9-0e96f20e3d54 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:29.002131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 159/310] 10.222.0.60 () {56 vars in 1308 bytes} [Mon Jul 25 12:26:28 2022] GET /identity/v3/role_assignments?user.id=3441a0a2da854eef9b3847539b2a8cc6&scope.system=all&role.id=06b987ffe3ed42d1bc1963d1459eaaa6 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:29.389034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef6591-beb9-4200-a963-598993958f61 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:29.389408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef6591-beb9-4200-a963-598993958f61 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:29.389609 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef6591-beb9-4200-a963-598993958f61 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:29.392197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 152/311] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:29.396381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009634d-1f8a-4ba6-afd5-aac62557b881 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:29.396673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009634d-1f8a-4ba6-afd5-aac62557b881 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:29.396954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009634d-1f8a-4ba6-afd5-aac62557b881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:29.415897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-8009634d-1f8a-4ba6-afd5-aac62557b881 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:29.417182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8009634d-1f8a-4ba6-afd5-aac62557b881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:29.471979 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:29.494135 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 160/312] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:29.517348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef6591-beb9-4200-a963-598993958f61 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:29.590337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:29.626019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef6591-beb9-4200-a963-598993958f61 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:29.661898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d956954-fe20-439b-83bc-4eb6d3830219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:29.662721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d956954-fe20-439b-83bc-4eb6d3830219 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:29.663001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d956954-fe20-439b-83bc-4eb6d3830219 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:29.663273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d956954-fe20-439b-83bc-4eb6d3830219 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:29.673843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d956954-fe20-439b-83bc-4eb6d3830219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-8d956954-fe20-439b-83bc-4eb6d3830219 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:29.682026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:29.683851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:29.684962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 153/313] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:29.689901 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009634d-1f8a-4ba6-afd5-aac62557b881 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:29.790330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009634d-1f8a-4ba6-afd5-aac62557b881 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:29.825129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7890b71-6c79-48d6-aae3-7f4a12e797eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:29.825801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7890b71-6c79-48d6-aae3-7f4a12e797eb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:29.826119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7890b71-6c79-48d6-aae3-7f4a12e797eb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:29.826530 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7890b71-6c79-48d6-aae3-7f4a12e797eb None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:29.827280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7890b71-6c79-48d6-aae3-7f4a12e797eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:29.827586 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7890b71-6c79-48d6-aae3-7f4a12e797eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:29.828961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7890b71-6c79-48d6-aae3-7f4a12e797eb None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:29.837433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 161/314] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:29.842697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d956954-fe20-439b-83bc-4eb6d3830219 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:29.949081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d956954-fe20-439b-83bc-4eb6d3830219 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:29.966395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:29.985773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1fc3b8-871b-4454-aa4f-359d564efc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:29.986809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fc3b8-871b-4454-aa4f-359d564efc03 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:29.987088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fc3b8-871b-4454-aa4f-359d564efc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:29.987362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fc3b8-871b-4454-aa4f-359d564efc03 None admin] PATH_INFO: `/v3/users/9ea5f0569ca74098b15427ab682fffde` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:29.988589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1fc3b8-871b-4454-aa4f-359d564efc03 None admin] RBAC: Authorizing `identity:get_user(user_id=9ea5f0569ca74098b15427ab682fffde)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:29.990737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1fc3b8-871b-4454-aa4f-359d564efc03 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:29.992945 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 154/315] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:29 2022] GET /identity/v3/users/9ea5f0569ca74098b15427ab682fffde => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:29.999020 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7890b71-6c79-48d6-aae3-7f4a12e797eb None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:30.045958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:30.104020 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7890b71-6c79-48d6-aae3-7f4a12e797eb None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:30.139311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246df98-1185-499f-8e3e-c2a38555edce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:30.140024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246df98-1185-499f-8e3e-c2a38555edce None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:30.140304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246df98-1185-499f-8e3e-c2a38555edce None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:30.140573 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246df98-1185-499f-8e3e-c2a38555edce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:30.217713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246df98-1185-499f-8e3e-c2a38555edce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:30.219213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246df98-1185-499f-8e3e-c2a38555edce None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:30.264954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:30.322199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 162/316] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:30.327292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1fc3b8-871b-4454-aa4f-359d564efc03 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:30.381626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bac390-d903-479d-a7ba-a99a79cdad86 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:30.381943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bac390-d903-479d-a7ba-a99a79cdad86 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:30.382217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bac390-d903-479d-a7ba-a99a79cdad86 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:30.383855 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 163/317] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:30.388781 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f30bf0-afcc-4753-9889-9decd6a26af3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:30.389075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f30bf0-afcc-4753-9889-9decd6a26af3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:30.389362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f30bf0-afcc-4753-9889-9decd6a26af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:30.408496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d8f30bf0-afcc-4753-9889-9decd6a26af3 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:30.409798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d8f30bf0-afcc-4753-9889-9decd6a26af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:30.438872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1fc3b8-871b-4454-aa4f-359d564efc03 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:30.483211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b2ec3-8d9b-4672-a54e-b36ff6e3b02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:30.484074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b2ec3-8d9b-4672-a54e-b36ff6e3b02d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:30.484357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b2ec3-8d9b-4672-a54e-b36ff6e3b02d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:30.484629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b2ec3-8d9b-4672-a54e-b36ff6e3b02d None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:30.485823 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b2ec3-8d9b-4672-a54e-b36ff6e3b02d None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:30.487285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b2ec3-8d9b-4672-a54e-b36ff6e3b02d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:30.489526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 155/318] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:30 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:30.491667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 164/319] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:30.495170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b2ec3-8d9b-4672-a54e-b36ff6e3b02d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:30.509272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f30bf0-afcc-4753-9889-9decd6a26af3 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:30.558226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:30.590553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:30.614973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b2ec3-8d9b-4672-a54e-b36ff6e3b02d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:30.618208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f30bf0-afcc-4753-9889-9decd6a26af3 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:30.651470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0feed6-2005-4cf5-a4f6-6c79b88434d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:30.652695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0feed6-2005-4cf5-a4f6-6c79b88434d6 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:30.652984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0feed6-2005-4cf5-a4f6-6c79b88434d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:30.653264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0feed6-2005-4cf5-a4f6-6c79b88434d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:30.654121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0feed6-2005-4cf5-a4f6-6c79b88434d6 None admin] RBAC: Adding query filter params (role.id=06b987ffe3ed42d1bc1963d1459eaaa6, scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e, user.id=9ea5f0569ca74098b15427ab682fffde) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:30.654438 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0feed6-2005-4cf5-a4f6-6c79b88434d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:30.654934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454bda9-c273-44da-ad3c-13e050183238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:30.655726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454bda9-c273-44da-ad3c-13e050183238 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:30.655887 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0feed6-2005-4cf5-a4f6-6c79b88434d6 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:30.656034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454bda9-c273-44da-ad3c-13e050183238 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:30.656268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454bda9-c273-44da-ad3c-13e050183238 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:30.664172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 156/320] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:30 2022] GET /identity/v3/role_assignments?user.id=9ea5f0569ca74098b15427ab682fffde&scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e&role.id=06b987ffe3ed42d1bc1963d1459eaaa6 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:30.665429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8063204-e54d-45cd-9e22-3b19f4e4a6d7 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:30.665712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8063204-e54d-45cd-9e22-3b19f4e4a6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:30.666013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8063204-e54d-45cd-9e22-3b19f4e4a6d7 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:30.667431 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 157/321] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:30.672928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75596ec3-8b5b-411e-8509-7d16f068a467 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:30.673209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75596ec3-8b5b-411e-8509-7d16f068a467 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:30.673490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75596ec3-8b5b-411e-8509-7d16f068a467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:30.677979 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454bda9-c273-44da-ad3c-13e050183238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-0454bda9-c273-44da-ad3c-13e050183238 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:30.689992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:30.691377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:30.692208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:30.692208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:30.692208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:30.692208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:30.692208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:30.692208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:30.692208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:30.692208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:30.692208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:30.692208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:30.692208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:30.692208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:30.693866 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-75596ec3-8b5b-411e-8509-7d16f068a467 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:30.695751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 165/322] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:30.695974 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-75596ec3-8b5b-411e-8509-7d16f068a467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:30.701722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454bda9-c273-44da-ad3c-13e050183238 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:30.784538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 158/323] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:30.806874 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75596ec3-8b5b-411e-8509-7d16f068a467 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:30.837895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454bda9-c273-44da-ad3c-13e050183238 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:30.861663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:30.883788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adef9f2-fe4a-41f2-98c3-bcfb242c075f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:30.884990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adef9f2-fe4a-41f2-98c3-bcfb242c075f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:30.884990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adef9f2-fe4a-41f2-98c3-bcfb242c075f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:30.884990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adef9f2-fe4a-41f2-98c3-bcfb242c075f None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:30.886301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adef9f2-fe4a-41f2-98c3-bcfb242c075f None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:30.886638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adef9f2-fe4a-41f2-98c3-bcfb242c075f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:30.888191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adef9f2-fe4a-41f2-98c3-bcfb242c075f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:30.889443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:30.898963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 166/324] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:30.904968 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adef9f2-fe4a-41f2-98c3-bcfb242c075f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:30.938502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75596ec3-8b5b-411e-8509-7d16f068a467 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:30.976549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d7aff-28d9-4fa6-9ddc-2a0d057bc83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:30.977352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d7aff-28d9-4fa6-9ddc-2a0d057bc83b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:30.977635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d7aff-28d9-4fa6-9ddc-2a0d057bc83b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:30.977941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d7aff-28d9-4fa6-9ddc-2a0d057bc83b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:30.979109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d7aff-28d9-4fa6-9ddc-2a0d057bc83b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:30.981334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d7aff-28d9-4fa6-9ddc-2a0d057bc83b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:30.983167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 159/325] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:30.988379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d7aff-28d9-4fa6-9ddc-2a0d057bc83b None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:31.015992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adef9f2-fe4a-41f2-98c3-bcfb242c075f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:31.052339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58702070-ee7d-426f-9a7b-da50bf1391f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:31.053215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58702070-ee7d-426f-9a7b-da50bf1391f7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:31.053503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58702070-ee7d-426f-9a7b-da50bf1391f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:31.053803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58702070-ee7d-426f-9a7b-da50bf1391f7 None admin] PATH_INFO: `/v3/users/23ea533b79dc4c88adce6095cbb8d950` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:31.055006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58702070-ee7d-426f-9a7b-da50bf1391f7 None admin] RBAC: Authorizing `identity:get_user(user_id=23ea533b79dc4c88adce6095cbb8d950)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:31.056439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58702070-ee7d-426f-9a7b-da50bf1391f7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:31.058284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 167/326] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:30 2022] GET /identity/v3/users/23ea533b79dc4c88adce6095cbb8d950 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:31.063888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58702070-ee7d-426f-9a7b-da50bf1391f7 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:31.100282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d7aff-28d9-4fa6-9ddc-2a0d057bc83b None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:31.114826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:31.127958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:31.135347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c1384-7cbe-48cf-b74e-121635c58f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:31.136041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c1384-7cbe-48cf-b74e-121635c58f0c None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:31.136327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c1384-7cbe-48cf-b74e-121635c58f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:31.136608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c1384-7cbe-48cf-b74e-121635c58f0c None admin] PATH_INFO: `/v3/groups` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:31.137305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c1384-7cbe-48cf-b74e-121635c58f0c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:31.138795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c1384-7cbe-48cf-b74e-121635c58f0c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:31.165942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 160/327] 10.222.0.60 () {60 vars in 1134 bytes} [Mon Jul 25 12:26:30 2022] POST /identity/v3/groups => generated 226 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:26:31.165942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e0512-b5f1-4b8e-ac34-f62ef81e5f72 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:31.165942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e0512-b5f1-4b8e-ac34-f62ef81e5f72 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:31.165942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e0512-b5f1-4b8e-ac34-f62ef81e5f72 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:31.165942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 161/328] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:31.165942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cecd85-631a-4bcf-951c-928cb813a99c None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:31.165942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cecd85-631a-4bcf-951c-928cb813a99c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:31.166507 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cecd85-631a-4bcf-951c-928cb813a99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:31.173892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58702070-ee7d-426f-9a7b-da50bf1391f7 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:31.183378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-c9cecd85-631a-4bcf-951c-928cb813a99c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:31.184646 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c9cecd85-631a-4bcf-951c-928cb813a99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:31.214902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3e4d7-0bfd-4648-bec3-69e6b206de4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:31.215589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3e4d7-0bfd-4648-bec3-69e6b206de4d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:31.215867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3e4d7-0bfd-4648-bec3-69e6b206de4d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:31.216141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3e4d7-0bfd-4648-bec3-69e6b206de4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:31.216995 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3e4d7-0bfd-4648-bec3-69e6b206de4d None admin] RBAC: Adding query filter params (role.id=86e07561a0854fc28db2e9f357e53d66, user.id=23ea533b79dc4c88adce6095cbb8d950, scope.system=all) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:31.217297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3e4d7-0bfd-4648-bec3-69e6b206de4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:31.218798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3e4d7-0bfd-4648-bec3-69e6b206de4d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:31.241947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 168/329] 10.222.0.60 () {56 vars in 1308 bytes} [Mon Jul 25 12:26:31 2022] GET /identity/v3/role_assignments?user.id=23ea533b79dc4c88adce6095cbb8d950&scope.system=all&role.id=86e07561a0854fc28db2e9f357e53d66 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:31.275014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 162/330] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:31.298908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3e4d7-0bfd-4648-bec3-69e6b206de4d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:31.414102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3e4d7-0bfd-4648-bec3-69e6b206de4d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:31.438446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:31.452536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5b27f-da55-4d9d-87f8-fc2bb014d40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:31.453340 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5b27f-da55-4d9d-87f8-fc2bb014d40e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:31.453619 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5b27f-da55-4d9d-87f8-fc2bb014d40e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:31.453889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5b27f-da55-4d9d-87f8-fc2bb014d40e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:31.465290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5b27f-da55-4d9d-87f8-fc2bb014d40e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-fcf5b27f-da55-4d9d-87f8-fc2bb014d40e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:31.473221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:31.474337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:31.475302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 169/331] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:31.480464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cecd85-631a-4bcf-951c-928cb813a99c None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:31.591666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cecd85-631a-4bcf-951c-928cb813a99c None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:31.594938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:31.633029 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f325b-38ea-4938-8b24-178e9eb32669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:31.633700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f325b-38ea-4938-8b24-178e9eb32669 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:31.634010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f325b-38ea-4938-8b24-178e9eb32669 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:31.634290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f325b-38ea-4938-8b24-178e9eb32669 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:31.635045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f325b-38ea-4938-8b24-178e9eb32669 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:31.635344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f325b-38ea-4938-8b24-178e9eb32669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:31.637278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f325b-38ea-4938-8b24-178e9eb32669 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:31.646200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 163/332] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:31.652415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5b27f-da55-4d9d-87f8-fc2bb014d40e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:31.762672 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5b27f-da55-4d9d-87f8-fc2bb014d40e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:31.797627 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a286e5-b55a-4191-8ff2-6184af2c8247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:31.798580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a286e5-b55a-4191-8ff2-6184af2c8247 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:31.798844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a286e5-b55a-4191-8ff2-6184af2c8247 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:31.799117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a286e5-b55a-4191-8ff2-6184af2c8247 None admin] PATH_INFO: `/v3/users/3441a0a2da854eef9b3847539b2a8cc6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:31.800337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a286e5-b55a-4191-8ff2-6184af2c8247 None admin] RBAC: Authorizing `identity:get_user(user_id=3441a0a2da854eef9b3847539b2a8cc6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:31.801808 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a286e5-b55a-4191-8ff2-6184af2c8247 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:31.803953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 170/333] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:31 2022] GET /identity/v3/users/3441a0a2da854eef9b3847539b2a8cc6 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:31.810123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f325b-38ea-4938-8b24-178e9eb32669 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:31.929699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f325b-38ea-4938-8b24-178e9eb32669 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:31.965869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dafb9e-43a9-482c-894c-f29a49e4706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:31.967011 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dafb9e-43a9-482c-894c-f29a49e4706d None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:31.967437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dafb9e-43a9-482c-894c-f29a49e4706d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:31.967881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dafb9e-43a9-482c-894c-f29a49e4706d None admin] PATH_INFO: `/v3/system/users/3441a0a2da854eef9b3847539b2a8cc6/roles/06b987ffe3ed42d1bc1963d1459eaaa6` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:31.969550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dafb9e-43a9-482c-894c-f29a49e4706d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3441a0a2da854eef9b3847539b2a8cc6, role_id=06b987ffe3ed42d1bc1963d1459eaaa6)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:31.971155 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dafb9e-43a9-482c-894c-f29a49e4706d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:31.984908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 164/334] 10.222.0.60 () {58 vars in 1334 bytes} [Mon Jul 25 12:26:31 2022] PUT /identity/v3/system/users/3441a0a2da854eef9b3847539b2a8cc6/roles/06b987ffe3ed42d1bc1963d1459eaaa6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:33.607077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9a4dd-3d01-4c83-a29f-2919c14d8304 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:33.608079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9a4dd-3d01-4c83-a29f-2919c14d8304 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:33.608584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9a4dd-3d01-4c83-a29f-2919c14d8304 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:33.610486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 171/335] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:33.616294 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e3fa1-eafe-4c3b-8ec5-f62092280a46 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:33.617109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e3fa1-eafe-4c3b-8ec5-f62092280a46 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:33.617595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e3fa1-eafe-4c3b-8ec5-f62092280a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:33.638660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-391e3fa1-eafe-4c3b-8ec5-f62092280a46 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:33.640203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-391e3fa1-eafe-4c3b-8ec5-f62092280a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:33.673668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:33.734031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 165/336] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:33.752880 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9a4dd-3d01-4c83-a29f-2919c14d8304 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:33.755232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bf8eb-5003-4abe-8284-6e82deefb8e0 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:33.755517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bf8eb-5003-4abe-8284-6e82deefb8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:33.755793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bf8eb-5003-4abe-8284-6e82deefb8e0 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:33.762146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 166/337] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:33.776232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428adfb-3ff9-4840-bb42-858fc1c3886b None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:33.777088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428adfb-3ff9-4840-bb42-858fc1c3886b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:33.777370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428adfb-3ff9-4840-bb42-858fc1c3886b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:33.804192 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-8428adfb-3ff9-4840-bb42-858fc1c3886b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:33.805756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8428adfb-3ff9-4840-bb42-858fc1c3886b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:33.879375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:33.903543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 167/338] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:33.945868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428adfb-3ff9-4840-bb42-858fc1c3886b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:34.011593 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9a4dd-3d01-4c83-a29f-2919c14d8304 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:34.087737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:34.098299 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428adfb-3ff9-4840-bb42-858fc1c3886b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:34.118625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f09ca-2f36-40a6-a2cc-7d8f8e0b177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:34.119897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f09ca-2f36-40a6-a2cc-7d8f8e0b177d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:34.119897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f09ca-2f36-40a6-a2cc-7d8f8e0b177d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:34.120647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f09ca-2f36-40a6-a2cc-7d8f8e0b177d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:34.156706 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f09ca-2f36-40a6-a2cc-7d8f8e0b177d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:34.173941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edb0af-a142-4b06-8499-653e2e228d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:34.173941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edb0af-a142-4b06-8499-653e2e228d14 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:34.173941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edb0af-a142-4b06-8499-653e2e228d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:34.173941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edb0af-a142-4b06-8499-653e2e228d14 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-e31f09ca-2f36-40a6-a2cc-7d8f8e0b177d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:34.179301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:34.180176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:34.180993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:34.180993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:34.180993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:34.180993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:34.180993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:34.180993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:34.180993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:34.180993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:34.180993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:34.180993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:34.180993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:34.180993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:34.183985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 172/339] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:34.194947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f09ca-2f36-40a6-a2cc-7d8f8e0b177d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:34.196638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edb0af-a142-4b06-8499-653e2e228d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-74edb0af-a142-4b06-8499-653e2e228d14 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:34.208980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:34.209882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:34.210703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:34.210703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:34.210703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:34.210703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:34.210703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:34.210703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:34.210703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:34.210703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:34.210703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:34.210703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:34.210703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:34.210703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:34.213549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 168/340] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:26:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:34.219251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edb0af-a142-4b06-8499-653e2e228d14 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:34.347051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f09ca-2f36-40a6-a2cc-7d8f8e0b177d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:34.361861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:34.390774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016dfb1-23a2-42cb-b46c-921b531c61ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:34.391447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016dfb1-23a2-42cb-b46c-921b531c61ae None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:34.391735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016dfb1-23a2-42cb-b46c-921b531c61ae None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:34.392018 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016dfb1-23a2-42cb-b46c-921b531c61ae None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:34.392773 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016dfb1-23a2-42cb-b46c-921b531c61ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:34.393084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016dfb1-23a2-42cb-b46c-921b531c61ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:34.394534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016dfb1-23a2-42cb-b46c-921b531c61ae None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:34.404826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 173/341] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:34.411252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016dfb1-23a2-42cb-b46c-921b531c61ae None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:34.415779 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edb0af-a142-4b06-8499-653e2e228d14 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:34.474746 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33901f0-45e1-4850-84b2-e63a7afc2e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:34.475865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33901f0-45e1-4850-84b2-e63a7afc2e54 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:34.476363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33901f0-45e1-4850-84b2-e63a7afc2e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:34.476872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33901f0-45e1-4850-84b2-e63a7afc2e54 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:34.477866 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33901f0-45e1-4850-84b2-e63a7afc2e54 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:34.478443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33901f0-45e1-4850-84b2-e63a7afc2e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:34.480936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33901f0-45e1-4850-84b2-e63a7afc2e54 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:34.492718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 169/342] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:34.498672 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33901f0-45e1-4850-84b2-e63a7afc2e54 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:34.520824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016dfb1-23a2-42cb-b46c-921b531c61ae None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:34.522474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:34.558434 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6632a-ca4f-4ebf-a78b-70947cb4ac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:34.559150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6632a-ca4f-4ebf-a78b-70947cb4ac3c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:34.559441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6632a-ca4f-4ebf-a78b-70947cb4ac3c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:34.559723 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6632a-ca4f-4ebf-a78b-70947cb4ac3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:34.606344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:34.633325 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6632a-ca4f-4ebf-a78b-70947cb4ac3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:34.634712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6632a-ca4f-4ebf-a78b-70947cb4ac3c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:34.673666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33901f0-45e1-4850-84b2-e63a7afc2e54 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:34.735212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cb12c-8c53-4cb9-b2bf-e8395487e974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:34.735881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cb12c-8c53-4cb9-b2bf-e8395487e974 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:34.736159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cb12c-8c53-4cb9-b2bf-e8395487e974 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:34.736428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cb12c-8c53-4cb9-b2bf-e8395487e974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:34.742102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 174/343] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:26:34.748459 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6632a-ca4f-4ebf-a78b-70947cb4ac3c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:34.830371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:34.858826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6632a-ca4f-4ebf-a78b-70947cb4ac3c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:34.867005 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470cb12c-8c53-4cb9-b2bf-e8395487e974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:34.868490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470cb12c-8c53-4cb9-b2bf-e8395487e974 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:34.899830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:34.910219 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c2d70-8254-402d-9d73-a34354c95aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:34.911097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c2d70-8254-402d-9d73-a34354c95aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:34.911381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c2d70-8254-402d-9d73-a34354c95aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:34.911663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c2d70-8254-402d-9d73-a34354c95aa3 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:34.916015 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c2d70-8254-402d-9d73-a34354c95aa3 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:34.919102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c2d70-8254-402d-9d73-a34354c95aa3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:34.922215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 175/344] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:34 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:34.923438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1beb9-c685-486c-81c0-82da359b6f3d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:34.923777 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1beb9-c685-486c-81c0-82da359b6f3d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:34.924009 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1beb9-c685-486c-81c0-82da359b6f3d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:34.934645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 176/345] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:34 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:34.940565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb712ae1-e831-4c76-bc18-e89bbaf32ecb None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:34.940860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb712ae1-e831-4c76-bc18-e89bbaf32ecb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:34.941140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb712ae1-e831-4c76-bc18-e89bbaf32ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:34.974160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-eb712ae1-e831-4c76-bc18-e89bbaf32ecb None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:34.975701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-eb712ae1-e831-4c76-bc18-e89bbaf32ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:35.022595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 170/346] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:26:35.023528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cb12c-8c53-4cb9-b2bf-e8395487e974 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:35.070033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 177/347] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:35.071758 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f400ee-4cec-4cb8-8782-358d8a86feeb None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:35.072248 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f400ee-4cec-4cb8-8782-358d8a86feeb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:35.072741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f400ee-4cec-4cb8-8782-358d8a86feeb None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:35.078598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 178/348] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:35.079582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f400ee-4cec-4cb8-8782-358d8a86feeb None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:35.133319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cb12c-8c53-4cb9-b2bf-e8395487e974 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:35.149830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:35.184694 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:35.186813 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da2259-7e01-4360-8df2-882249dfa0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:35.188016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da2259-7e01-4360-8df2-882249dfa0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:35.188490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da2259-7e01-4360-8df2-882249dfa0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:35.189692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da2259-7e01-4360-8df2-882249dfa0f3 None admin] PATH_INFO: `/v3/groups/0a2bd866b81249e7b752252ab4af5335` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:35.199145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da2259-7e01-4360-8df2-882249dfa0f3 None admin] RBAC: Authorizing `identity:get_group(group_id=0a2bd866b81249e7b752252ab4af5335)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:35.200905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da2259-7e01-4360-8df2-882249dfa0f3 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:35.203667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 171/349] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:35 2022] GET /identity/v3/groups/0a2bd866b81249e7b752252ab4af5335 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:35.205096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd3193-3797-4bac-a9e1-10b2ee7fadc4 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:35.205600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd3193-3797-4bac-a9e1-10b2ee7fadc4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:35.206229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd3193-3797-4bac-a9e1-10b2ee7fadc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:35.226036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-1afd3193-3797-4bac-a9e1-10b2ee7fadc4 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:35.227872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-1afd3193-3797-4bac-a9e1-10b2ee7fadc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:35.242514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f400ee-4cec-4cb8-8782-358d8a86feeb None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:35.291866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fa0f3-b0dd-42ed-b347-92f0ee0d50ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:35.293745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fa0f3-b0dd-42ed-b347-92f0ee0d50ed None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:35.294080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fa0f3-b0dd-42ed-b347-92f0ee0d50ed None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:35.294364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fa0f3-b0dd-42ed-b347-92f0ee0d50ed None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:35.297537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fa0f3-b0dd-42ed-b347-92f0ee0d50ed None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:35.298659 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fa0f3-b0dd-42ed-b347-92f0ee0d50ed None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:35.300812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 179/350] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:35 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:35.301587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fa0f3-b0dd-42ed-b347-92f0ee0d50ed None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:35.326445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 172/351] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:35.327880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd3193-3797-4bac-a9e1-10b2ee7fadc4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:35.438832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fa0f3-b0dd-42ed-b347-92f0ee0d50ed None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:35.452932 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd3193-3797-4bac-a9e1-10b2ee7fadc4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:35.460715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:35.482182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3da16-55a5-4281-9e05-9ed1f4c9f1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:35.483003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3da16-55a5-4281-9e05-9ed1f4c9f1dc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:35.483290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3da16-55a5-4281-9e05-9ed1f4c9f1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:35.483570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3da16-55a5-4281-9e05-9ed1f4c9f1dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:35.494022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:35.499470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3da16-55a5-4281-9e05-9ed1f4c9f1dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:35.503278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156473e-a485-4bac-ab40-3a260b1fec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:35.504307 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156473e-a485-4bac-ab40-3a260b1fec99 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:35.504591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156473e-a485-4bac-ab40-3a260b1fec99 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:35.504869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156473e-a485-4bac-ab40-3a260b1fec99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:35.505715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156473e-a485-4bac-ab40-3a260b1fec99 None admin] RBAC: Adding query filter params (group.id=0a2bd866b81249e7b752252ab4af5335, role.id=86e07561a0854fc28db2e9f357e53d66, scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:35.506059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156473e-a485-4bac-ab40-3a260b1fec99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:35.507531 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156473e-a485-4bac-ab40-3a260b1fec99 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c7e3da16-55a5-4281-9e05-9ed1f4c9f1dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:35.508610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:35.509480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:35.510310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 180/352] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:35.512097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3da16-55a5-4281-9e05-9ed1f4c9f1dc None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:35.519831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 173/353] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Jul 25 12:26:35 2022] GET /identity/v3/role_assignments?group.id=0a2bd866b81249e7b752252ab4af5335&scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1&role.id=86e07561a0854fc28db2e9f357e53d66 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:35.521249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41aa3e-21f4-41f0-9d62-c6cbf736ea30 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:35.521538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41aa3e-21f4-41f0-9d62-c6cbf736ea30 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:35.521812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41aa3e-21f4-41f0-9d62-c6cbf736ea30 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:35.523522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 174/354] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:35.524167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41aa3e-21f4-41f0-9d62-c6cbf736ea30 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:35.664462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3da16-55a5-4281-9e05-9ed1f4c9f1dc None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:35.679900 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41aa3e-21f4-41f0-9d62-c6cbf736ea30 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:35.707848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb95c7-b32e-465e-89d7-0852b58178a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:35.708666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb95c7-b32e-465e-89d7-0852b58178a7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:35.709017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb95c7-b32e-465e-89d7-0852b58178a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:35.709530 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb95c7-b32e-465e-89d7-0852b58178a7 None admin] PATH_INFO: `/v3/groups/0a2bd866b81249e7b752252ab4af5335` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:35.712506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb95c7-b32e-465e-89d7-0852b58178a7 None admin] RBAC: Authorizing `identity:get_group(group_id=0a2bd866b81249e7b752252ab4af5335)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:35.714522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb95c7-b32e-465e-89d7-0852b58178a7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:35.717406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 181/355] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:35 2022] GET /identity/v3/groups/0a2bd866b81249e7b752252ab4af5335 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:35.718035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb95c7-b32e-465e-89d7-0852b58178a7 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:35.726325 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bbfff-4808-4ad5-9fd9-1966f2804065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:35.727142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bbfff-4808-4ad5-9fd9-1966f2804065 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:35.727423 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bbfff-4808-4ad5-9fd9-1966f2804065 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:35.727700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bbfff-4808-4ad5-9fd9-1966f2804065 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:35.742146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bbfff-4808-4ad5-9fd9-1966f2804065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-9c7bbfff-4808-4ad5-9fd9-1966f2804065 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:35.751718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:35.752640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:35.753424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 175/356] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:26:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:35.754692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62f862-ae7a-4a0d-b47e-e16fbd9c7c9e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:35.754692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62f862-ae7a-4a0d-b47e-e16fbd9c7c9e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:35.754927 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62f862-ae7a-4a0d-b47e-e16fbd9c7c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:35.777898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-3e62f862-ae7a-4a0d-b47e-e16fbd9c7c9e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:35.779263 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-3e62f862-ae7a-4a0d-b47e-e16fbd9c7c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:35.807799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:35.808873 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:35.860565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb95c7-b32e-465e-89d7-0852b58178a7 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:35.864696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 176/357] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:35.866905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62f862-ae7a-4a0d-b47e-e16fbd9c7c9e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:35.899700 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08512cc8-7faa-4059-a175-3f62ccb6a397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:35.900385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08512cc8-7faa-4059-a175-3f62ccb6a397 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:35.900669 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08512cc8-7faa-4059-a175-3f62ccb6a397 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:35.901060 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08512cc8-7faa-4059-a175-3f62ccb6a397 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:35.902513 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08512cc8-7faa-4059-a175-3f62ccb6a397 None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:35.902823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08512cc8-7faa-4059-a175-3f62ccb6a397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:35.904212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08512cc8-7faa-4059-a175-3f62ccb6a397 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:35.920317 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 182/358] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:35.922096 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff7fa6-898b-4a9f-95aa-c96898574ae7 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:35.922383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff7fa6-898b-4a9f-95aa-c96898574ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:35.922661 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff7fa6-898b-4a9f-95aa-c96898574ae7 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:35.923908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 183/359] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:35.925310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff7fa6-898b-4a9f-95aa-c96898574ae7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:35.997432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62f862-ae7a-4a0d-b47e-e16fbd9c7c9e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:36.039028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfac53b-ccbc-4e99-9432-fc727764a562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:36.039695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfac53b-ccbc-4e99-9432-fc727764a562 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:36.039977 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfac53b-ccbc-4e99-9432-fc727764a562 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:36.040259 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfac53b-ccbc-4e99-9432-fc727764a562 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:36.041101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfac53b-ccbc-4e99-9432-fc727764a562 None admin] RBAC: Adding query filter params (group.id=0a2bd866b81249e7b752252ab4af5335, role.id=950510c081de4ab6b12f383c0748450d, scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:36.041411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfac53b-ccbc-4e99-9432-fc727764a562 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:36.042913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfac53b-ccbc-4e99-9432-fc727764a562 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:36.049988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 177/360] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Jul 25 12:26:35 2022] GET /identity/v3/role_assignments?group.id=0a2bd866b81249e7b752252ab4af5335&scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1&role.id=950510c081de4ab6b12f383c0748450d => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:36.051740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfac53b-ccbc-4e99-9432-fc727764a562 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:36.059812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff7fa6-898b-4a9f-95aa-c96898574ae7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:36.086669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:36.098395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:36.108316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f44755-81cc-4422-bb83-51170fec3ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:36.108999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f44755-81cc-4422-bb83-51170fec3ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:36.109284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f44755-81cc-4422-bb83-51170fec3ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:36.109561 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f44755-81cc-4422-bb83-51170fec3ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:36.110368 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f44755-81cc-4422-bb83-51170fec3ab8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:36.110675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f44755-81cc-4422-bb83-51170fec3ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:36.112390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f44755-81cc-4422-bb83-51170fec3ab8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:36.126728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 184/361] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:36.126728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f44755-81cc-4422-bb83-51170fec3ab8 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:36.199303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfac53b-ccbc-4e99-9432-fc727764a562 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:36.246019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5a68f-d04b-4a01-a484-8f3ff63bfe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:36.247386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5a68f-d04b-4a01-a484-8f3ff63bfe0a None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:36.247919 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5a68f-d04b-4a01-a484-8f3ff63bfe0a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:36.248386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5a68f-d04b-4a01-a484-8f3ff63bfe0a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:36.262658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5a68f-d04b-4a01-a484-8f3ff63bfe0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-f9b5a68f-d04b-4a01-a484-8f3ff63bfe0a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:36.273255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:36.274161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:36.274941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:36.274941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:36.274941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:36.274941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:36.274941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:36.274941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:36.274941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:36.274941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:36.274941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:36.274941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:36.274941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:26:36.274941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:36.275538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f44755-81cc-4422-bb83-51170fec3ab8 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:36.278377 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 178/362] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:36.279841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b3be9-917a-4351-a24b-c1be4f4c4fa8 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:36.280322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b3be9-917a-4351-a24b-c1be4f4c4fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:36.280791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b3be9-917a-4351-a24b-c1be4f4c4fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:36.305386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-675b3be9-917a-4351-a24b-c1be4f4c4fa8 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:36.307413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-675b3be9-917a-4351-a24b-c1be4f4c4fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:36.313186 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078808f-8eb2-4d0a-a8f4-c818dcc324fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:36.313842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078808f-8eb2-4d0a-a8f4-c818dcc324fd None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:36.316248 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078808f-8eb2-4d0a-a8f4-c818dcc324fd None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:36.316533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078808f-8eb2-4d0a-a8f4-c818dcc324fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:36.386738 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:36.398862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078808f-8eb2-4d0a-a8f4-c818dcc324fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:36.399177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 179/363] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:36.400269 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b3be9-917a-4351-a24b-c1be4f4c4fa8 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:36.400546 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078808f-8eb2-4d0a-a8f4-c818dcc324fd None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:36.419690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:36.521381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 185/364] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:36.521381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078808f-8eb2-4d0a-a8f4-c818dcc324fd None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:36.538427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b3be9-917a-4351-a24b-c1be4f4c4fa8 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:36.596735 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d89d17-f6b9-41c1-8e38-969e520c86a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:36.597401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d89d17-f6b9-41c1-8e38-969e520c86a2 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:36.597685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d89d17-f6b9-41c1-8e38-969e520c86a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:36.597989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d89d17-f6b9-41c1-8e38-969e520c86a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:36.632809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:36.644766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078808f-8eb2-4d0a-a8f4-c818dcc324fd None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:36.672014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d89d17-f6b9-41c1-8e38-969e520c86a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:36.673487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d89d17-f6b9-41c1-8e38-969e520c86a2 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:36.681041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e97f5-9cbb-47fb-b995-b95aabc0fa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:36.681716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e97f5-9cbb-47fb-b995-b95aabc0fa18 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:36.682041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e97f5-9cbb-47fb-b995-b95aabc0fa18 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:36.682317 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e97f5-9cbb-47fb-b995-b95aabc0fa18 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:36.683638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e97f5-9cbb-47fb-b995-b95aabc0fa18 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:36.683945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e97f5-9cbb-47fb-b995-b95aabc0fa18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:36.685329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e97f5-9cbb-47fb-b995-b95aabc0fa18 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:36.692523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:36.694601 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 186/365] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:36.695609 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e97f5-9cbb-47fb-b995-b95aabc0fa18 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:36.790686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 180/366] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:36.791558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d89d17-f6b9-41c1-8e38-969e520c86a2 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:36.805411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e97f5-9cbb-47fb-b995-b95aabc0fa18 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:36.848181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117f4d5-eaac-4f7d-9711-8e0c7bd77d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:36.849140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117f4d5-eaac-4f7d-9711-8e0c7bd77d8d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:36.849443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117f4d5-eaac-4f7d-9711-8e0c7bd77d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:36.849793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117f4d5-eaac-4f7d-9711-8e0c7bd77d8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:36.863803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117f4d5-eaac-4f7d-9711-8e0c7bd77d8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a117f4d5-eaac-4f7d-9711-8e0c7bd77d8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:36.872569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:36.873471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:36.874502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 187/367] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:26:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:36.875315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117f4d5-eaac-4f7d-9711-8e0c7bd77d8d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:36.898753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:36.904802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d89d17-f6b9-41c1-8e38-969e520c86a2 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:36.920439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:36.948600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6735566-18f3-4643-b519-ab002bf189c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:36.949845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6735566-18f3-4643-b519-ab002bf189c1 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:36.950167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6735566-18f3-4643-b519-ab002bf189c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:36.950457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6735566-18f3-4643-b519-ab002bf189c1 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:36.951644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6735566-18f3-4643-b519-ab002bf189c1 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:36.956162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6735566-18f3-4643-b519-ab002bf189c1 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:36.956162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 181/368] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:36 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:36.956162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6735566-18f3-4643-b519-ab002bf189c1 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:37.022782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117f4d5-eaac-4f7d-9711-8e0c7bd77d8d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:37.065187 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1fcfc-2396-4efd-8dc0-1d5afafeb77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:37.066511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1fcfc-2396-4efd-8dc0-1d5afafeb77f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:37.067013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1fcfc-2396-4efd-8dc0-1d5afafeb77f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:37.067511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1fcfc-2396-4efd-8dc0-1d5afafeb77f None admin] PATH_INFO: `/v3/users/3441a0a2da854eef9b3847539b2a8cc6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:37.068911 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1fcfc-2396-4efd-8dc0-1d5afafeb77f None admin] RBAC: Authorizing `identity:get_user(user_id=3441a0a2da854eef9b3847539b2a8cc6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:37.071633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1fcfc-2396-4efd-8dc0-1d5afafeb77f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:37.073895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 188/369] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:26:36 2022] GET /identity/v3/users/3441a0a2da854eef9b3847539b2a8cc6 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:37.074980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1fcfc-2396-4efd-8dc0-1d5afafeb77f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:37.095899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6735566-18f3-4643-b519-ab002bf189c1 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:37.142229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8874f77-6a66-440a-bafb-a1982b407ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:37.146540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8874f77-6a66-440a-bafb-a1982b407ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:37.147056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8874f77-6a66-440a-bafb-a1982b407ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:37.147554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8874f77-6a66-440a-bafb-a1982b407ce8 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:37.148991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8874f77-6a66-440a-bafb-a1982b407ce8 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:37.150667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8874f77-6a66-440a-bafb-a1982b407ce8 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:37.153651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 182/370] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:36 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:37.154739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8874f77-6a66-440a-bafb-a1982b407ce8 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:37.172824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:37.202904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1fcfc-2396-4efd-8dc0-1d5afafeb77f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:37.222205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:37.252030 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56de77-3b1b-4de5-a2b5-387172b180c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:37.252942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56de77-3b1b-4de5-a2b5-387172b180c5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:37.253385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56de77-3b1b-4de5-a2b5-387172b180c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:37.254177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56de77-3b1b-4de5-a2b5-387172b180c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:37.254967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56de77-3b1b-4de5-a2b5-387172b180c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:37.255283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56de77-3b1b-4de5-a2b5-387172b180c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:37.256681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56de77-3b1b-4de5-a2b5-387172b180c5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:37.277537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 189/371] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:37.279105 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56de77-3b1b-4de5-a2b5-387172b180c5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:37.293956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8874f77-6a66-440a-bafb-a1982b407ce8 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:37.333411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29cd19-86b7-4912-ba37-d37039006123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:37.334250 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29cd19-86b7-4912-ba37-d37039006123 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:37.334535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29cd19-86b7-4912-ba37-d37039006123 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:37.334810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29cd19-86b7-4912-ba37-d37039006123 None admin] PATH_INFO: `/v3/groups/0a2bd866b81249e7b752252ab4af5335` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:37.336094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29cd19-86b7-4912-ba37-d37039006123 None admin] RBAC: Authorizing `identity:get_group(group_id=0a2bd866b81249e7b752252ab4af5335)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:37.337575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29cd19-86b7-4912-ba37-d37039006123 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:37.340897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 183/372] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:37 2022] GET /identity/v3/groups/0a2bd866b81249e7b752252ab4af5335 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:37.341661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29cd19-86b7-4912-ba37-d37039006123 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:37.420677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56de77-3b1b-4de5-a2b5-387172b180c5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:37.457673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29cd19-86b7-4912-ba37-d37039006123 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:37.469875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ce431-d7c3-4fb4-8ade-c8910090cb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:37.471338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ce431-d7c3-4fb4-8ade-c8910090cb7d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:37.472039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ce431-d7c3-4fb4-8ade-c8910090cb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:37.472514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ce431-d7c3-4fb4-8ade-c8910090cb7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:37.473557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ce431-d7c3-4fb4-8ade-c8910090cb7d None admin] RBAC: Adding query filter params (role.id=06b987ffe3ed42d1bc1963d1459eaaa6, user.id=3441a0a2da854eef9b3847539b2a8cc6, scope.system=all) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:37.474221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ce431-d7c3-4fb4-8ade-c8910090cb7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:37.476175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ce431-d7c3-4fb4-8ade-c8910090cb7d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:37.481795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:37.489937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:37.501106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 190/373] 10.222.0.60 () {56 vars in 1308 bytes} [Mon Jul 25 12:26:37 2022] GET /identity/v3/role_assignments?user.id=3441a0a2da854eef9b3847539b2a8cc6&scope.system=all&role.id=06b987ffe3ed42d1bc1963d1459eaaa6 => generated 510 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:37.503087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ce431-d7c3-4fb4-8ade-c8910090cb7d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:37.506098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a721b-f7e2-4ed9-a0b3-cdbe58fa3e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:37.507073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a721b-f7e2-4ed9-a0b3-cdbe58fa3e6c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:37.507422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a721b-f7e2-4ed9-a0b3-cdbe58fa3e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:37.507812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a721b-f7e2-4ed9-a0b3-cdbe58fa3e6c None admin] PATH_INFO: `/v3/groups/0a2bd866b81249e7b752252ab4af5335` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:37.509257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a721b-f7e2-4ed9-a0b3-cdbe58fa3e6c None admin] RBAC: Authorizing `identity:get_group(group_id=0a2bd866b81249e7b752252ab4af5335)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:37.511937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a721b-f7e2-4ed9-a0b3-cdbe58fa3e6c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:37.513963 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 184/374] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:37 2022] GET /identity/v3/groups/0a2bd866b81249e7b752252ab4af5335 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:37.514937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a721b-f7e2-4ed9-a0b3-cdbe58fa3e6c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:37.620971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ce431-d7c3-4fb4-8ade-c8910090cb7d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:37.636467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a721b-f7e2-4ed9-a0b3-cdbe58fa3e6c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:37.660118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744dabbd-3058-4c0d-82c9-03e715ffdb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:37.661118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dabbd-3058-4c0d-82c9-03e715ffdb25 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:37.661580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dabbd-3058-4c0d-82c9-03e715ffdb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:37.662113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dabbd-3058-4c0d-82c9-03e715ffdb25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:37.676969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bad55-8b3e-437f-9fd3-d21c7c0762d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:37.677690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bad55-8b3e-437f-9fd3-d21c7c0762d2 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:37.678010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bad55-8b3e-437f-9fd3-d21c7c0762d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:37.678291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bad55-8b3e-437f-9fd3-d21c7c0762d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:37.679319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bad55-8b3e-437f-9fd3-d21c7c0762d2 None admin] RBAC: Adding query filter params (group.id=0a2bd866b81249e7b752252ab4af5335, role.id=86e07561a0854fc28db2e9f357e53d66, scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:37.679638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bad55-8b3e-437f-9fd3-d21c7c0762d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:37.681256 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bad55-8b3e-437f-9fd3-d21c7c0762d2 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:37.688711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 185/375] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Jul 25 12:26:37 2022] GET /identity/v3/role_assignments?group.id=0a2bd866b81249e7b752252ab4af5335&scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e&role.id=86e07561a0854fc28db2e9f357e53d66 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:37.689662 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bad55-8b3e-437f-9fd3-d21c7c0762d2 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:37.738626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:37.741568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744dabbd-3058-4c0d-82c9-03e715ffdb25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:37.743515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744dabbd-3058-4c0d-82c9-03e715ffdb25 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:37.758436 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:37.824643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bad55-8b3e-437f-9fd3-d21c7c0762d2 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:37.864601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5076019-c003-4b97-bcbb-01c8a9e7e39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:37.865364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5076019-c003-4b97-bcbb-01c8a9e7e39e None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:37.865656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5076019-c003-4b97-bcbb-01c8a9e7e39e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:37.866026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5076019-c003-4b97-bcbb-01c8a9e7e39e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:37.866885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5076019-c003-4b97-bcbb-01c8a9e7e39e None admin] RBAC: Adding query filter params (group.id=0a2bd866b81249e7b752252ab4af5335, role.id=950510c081de4ab6b12f383c0748450d, scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:37.867200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5076019-c003-4b97-bcbb-01c8a9e7e39e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:37.868700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5076019-c003-4b97-bcbb-01c8a9e7e39e None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:37.877338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 186/376] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Jul 25 12:26:37 2022] GET /identity/v3/role_assignments?group.id=0a2bd866b81249e7b752252ab4af5335&scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e&role.id=950510c081de4ab6b12f383c0748450d => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:37.877953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 191/377] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:37.878263 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5076019-c003-4b97-bcbb-01c8a9e7e39e None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:37.967022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b92fe2-7ee2-441f-96ff-073119fbdd75 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:37.967679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b92fe2-7ee2-441f-96ff-073119fbdd75 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:37.968203 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b92fe2-7ee2-441f-96ff-073119fbdd75 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:37.970188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 192/378] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:26:37.976829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51c174-09ae-424d-8823-1e5d56a576a8 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:37.977126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51c174-09ae-424d-8823-1e5d56a576a8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:37.977394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51c174-09ae-424d-8823-1e5d56a576a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:38.000639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5076019-c003-4b97-bcbb-01c8a9e7e39e None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:38.017173 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:38.037414 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1c51c174-09ae-424d-8823-1e5d56a576a8 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:38.038901 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1c51c174-09ae-424d-8823-1e5d56a576a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:38.043495 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fc520-0860-4234-9155-82151bdb7d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:38.044431 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fc520-0860-4234-9155-82151bdb7d28 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:38.044966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fc520-0860-4234-9155-82151bdb7d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:38.045252 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fc520-0860-4234-9155-82151bdb7d28 None admin] PATH_INFO: `/v3/projects/61157336119141239b389248fadfbdf0` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:38.046820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fc520-0860-4234-9155-82151bdb7d28 None admin] RBAC: Authorizing `identity:get_project(project_id=61157336119141239b389248fadfbdf0)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:38.048362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fc520-0860-4234-9155-82151bdb7d28 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:38.050372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 187/379] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:37 2022] GET /identity/v3/projects/61157336119141239b389248fadfbdf0 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:38.057254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fc520-0860-4234-9155-82151bdb7d28 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:38.159273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 193/380] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:38.183569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51c174-09ae-424d-8823-1e5d56a576a8 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:38.215023 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fc520-0860-4234-9155-82151bdb7d28 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:38.224090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:38.251848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3040e-cb72-431e-a2ab-034385626dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:38.252722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3040e-cb72-431e-a2ab-034385626dba None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:38.253018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3040e-cb72-431e-a2ab-034385626dba None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:38.253296 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3040e-cb72-431e-a2ab-034385626dba None admin] PATH_INFO: `/v3/groups/7b86e8287a634531a9027197ca3d0ac4` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:38.258527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a3040e-cb72-431e-a2ab-034385626dba None admin] RBAC: Authorizing `identity:get_group(group_id=7b86e8287a634531a9027197ca3d0ac4)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:38.260098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a3040e-cb72-431e-a2ab-034385626dba None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:38.266043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 188/381] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:38 2022] GET /identity/v3/groups/7b86e8287a634531a9027197ca3d0ac4 => generated 229 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:38.268694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3040e-cb72-431e-a2ab-034385626dba None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:38.301387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51c174-09ae-424d-8823-1e5d56a576a8 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:38.325745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:38.340179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edbb4a-c120-4dcb-87fb-69ca6f3c8a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:38.340988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edbb4a-c120-4dcb-87fb-69ca6f3c8a63 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:38.341267 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edbb4a-c120-4dcb-87fb-69ca6f3c8a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:38.341590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edbb4a-c120-4dcb-87fb-69ca6f3c8a63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:38.354021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edbb4a-c120-4dcb-87fb-69ca6f3c8a63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-41edbb4a-c120-4dcb-87fb-69ca6f3c8a63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:38.364554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:38.365518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:38.366341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:38.366341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:38.366341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:38.366341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:38.366341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:38.366341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:38.366341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:38.366341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:38.366341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:38.366341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:38.366341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:38.366341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:38.367440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 194/382] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:38.373817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edbb4a-c120-4dcb-87fb-69ca6f3c8a63 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:38.409528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3040e-cb72-431e-a2ab-034385626dba None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:38.482380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:38.490919 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edbb4a-c120-4dcb-87fb-69ca6f3c8a63 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:38.493767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0676ee-bd17-4c09-b925-030bbe1804c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:38.494762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0676ee-bd17-4c09-b925-030bbe1804c9 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:38.495790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0676ee-bd17-4c09-b925-030bbe1804c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:38.496288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0676ee-bd17-4c09-b925-030bbe1804c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:38.497339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0676ee-bd17-4c09-b925-030bbe1804c9 None admin] RBAC: Adding query filter params (group.id=7b86e8287a634531a9027197ca3d0ac4, role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=61157336119141239b389248fadfbdf0) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:38.497888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0676ee-bd17-4c09-b925-030bbe1804c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:38.499574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0676ee-bd17-4c09-b925-030bbe1804c9 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:38.516212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 189/383] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Jul 25 12:26:38 2022] GET /identity/v3/role_assignments?group.id=7b86e8287a634531a9027197ca3d0ac4&scope.project.id=61157336119141239b389248fadfbdf0&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 260 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:38.529188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4fb4d-0fe0-422c-a01f-109af5e3e8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:38.529947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4fb4d-0fe0-422c-a01f-109af5e3e8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:38.530253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4fb4d-0fe0-422c-a01f-109af5e3e8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:38.530531 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4fb4d-0fe0-422c-a01f-109af5e3e8a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:38.531309 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4fb4d-0fe0-422c-a01f-109af5e3e8a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:38.531617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4fb4d-0fe0-422c-a01f-109af5e3e8a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:38.533106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4fb4d-0fe0-422c-a01f-109af5e3e8a6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:38.543129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 195/384] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:38.548583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0676ee-bd17-4c09-b925-030bbe1804c9 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:38.661819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0676ee-bd17-4c09-b925-030bbe1804c9 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:38.674558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:38.705535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc56c9-a5d4-4853-b1b1-7d3a49bfce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:38.706455 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc56c9-a5d4-4853-b1b1-7d3a49bfce5d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:38.706746 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc56c9-a5d4-4853-b1b1-7d3a49bfce5d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:38.707029 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc56c9-a5d4-4853-b1b1-7d3a49bfce5d None admin] PATH_INFO: `/v3/groups/0a2bd866b81249e7b752252ab4af5335` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:38.710725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc56c9-a5d4-4853-b1b1-7d3a49bfce5d None admin] RBAC: Authorizing `identity:get_group(group_id=0a2bd866b81249e7b752252ab4af5335)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:38.712275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc56c9-a5d4-4853-b1b1-7d3a49bfce5d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:38.714137 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dae338-7624-474a-9501-0380ba3a7b79 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:38.714430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 190/385] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:38 2022] GET /identity/v3/groups/0a2bd866b81249e7b752252ab4af5335 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:38.714844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dae338-7624-474a-9501-0380ba3a7b79 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:38.715348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dae338-7624-474a-9501-0380ba3a7b79 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:38.716908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 196/386] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:38.722852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc56c9-a5d4-4853-b1b1-7d3a49bfce5d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:38.730265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cb4ed-ba83-47c2-8907-bfe825d690e7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:38.731184 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cb4ed-ba83-47c2-8907-bfe825d690e7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:38.731655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cb4ed-ba83-47c2-8907-bfe825d690e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:38.769858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-715cb4ed-ba83-47c2-8907-bfe825d690e7 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:38.771779 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-715cb4ed-ba83-47c2-8907-bfe825d690e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:38.866608 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 197/387] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:38.894055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc56c9-a5d4-4853-b1b1-7d3a49bfce5d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:38.898085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cb4ed-ba83-47c2-8907-bfe825d690e7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:38.978749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407570c6-34f9-41cc-bcab-3f4528b89514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:38.980137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407570c6-34f9-41cc-bcab-3f4528b89514 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:38.980426 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407570c6-34f9-41cc-bcab-3f4528b89514 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:38.980944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407570c6-34f9-41cc-bcab-3f4528b89514 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:38.982238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407570c6-34f9-41cc-bcab-3f4528b89514 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:38.983708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407570c6-34f9-41cc-bcab-3f4528b89514 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:38.985798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 191/388] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:38 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:38.990960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407570c6-34f9-41cc-bcab-3f4528b89514 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:39.024372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:39.048111 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cb4ed-ba83-47c2-8907-bfe825d690e7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:39.091077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea1ae-c632-409b-9907-d8fab60038fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:39.092368 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea1ae-c632-409b-9907-d8fab60038fb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:39.092656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea1ae-c632-409b-9907-d8fab60038fb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:39.092931 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea1ae-c632-409b-9907-d8fab60038fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:39.104156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cea1ae-c632-409b-9907-d8fab60038fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:39.113065 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-d2cea1ae-c632-409b-9907-d8fab60038fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:39.115452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:39.118029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:39.118810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 198/389] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:26:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:39.122879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea1ae-c632-409b-9907-d8fab60038fb None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:39.238641 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea1ae-c632-409b-9907-d8fab60038fb None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:39.276024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5bae7-c5a4-4bf1-8666-f055625e516c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:39.277274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5bae7-c5a4-4bf1-8666-f055625e516c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:39.277551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5bae7-c5a4-4bf1-8666-f055625e516c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:39.277818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5bae7-c5a4-4bf1-8666-f055625e516c None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:39.278614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5bae7-c5a4-4bf1-8666-f055625e516c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:39.279552 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5bae7-c5a4-4bf1-8666-f055625e516c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:39.281007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5bae7-c5a4-4bf1-8666-f055625e516c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:39.291790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 199/390] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:39.295812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407570c6-34f9-41cc-bcab-3f4528b89514 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:39.298712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5bae7-c5a4-4bf1-8666-f055625e516c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:39.303025 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:39.411489 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5bae7-c5a4-4bf1-8666-f055625e516c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:39.417547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b9e8d-8959-4ad9-86a6-cdf375c5aa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:39.418533 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b9e8d-8959-4ad9-86a6-cdf375c5aa3f None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:39.418822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b9e8d-8959-4ad9-86a6-cdf375c5aa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:39.419096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b9e8d-8959-4ad9-86a6-cdf375c5aa3f None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1/groups/0a2bd866b81249e7b752252ab4af5335/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:39.426747 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b9e8d-8959-4ad9-86a6-cdf375c5aa3f None admin] RBAC: Authorizing `identity:create_grant(project_id=e080e6f49f52471c81ff5d29b80d3bb1, group_id=0a2bd866b81249e7b752252ab4af5335, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:39.428557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b9e8d-8959-4ad9-86a6-cdf375c5aa3f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:39.448823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3c15d-a883-44ca-b262-295f8faab04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:39.449871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3c15d-a883-44ca-b262-295f8faab04e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:39.450183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3c15d-a883-44ca-b262-295f8faab04e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:39.450454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3c15d-a883-44ca-b262-295f8faab04e None admin] PATH_INFO: `/v3/groups/0a2bd866b81249e7b752252ab4af5335` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:39.451860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d3c15d-a883-44ca-b262-295f8faab04e None admin] RBAC: Authorizing `identity:get_group(group_id=0a2bd866b81249e7b752252ab4af5335)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:39.454337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d3c15d-a883-44ca-b262-295f8faab04e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:39.456141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 192/391] 10.222.0.60 () {58 vars in 1442 bytes} [Mon Jul 25 12:26:38 2022] PUT /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1/groups/0a2bd866b81249e7b752252ab4af5335/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:26:39.456331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 200/392] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:39 2022] GET /identity/v3/groups/0a2bd866b81249e7b752252ab4af5335 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:39.462147 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b9e8d-8959-4ad9-86a6-cdf375c5aa3f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:39.594733 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b9e8d-8959-4ad9-86a6-cdf375c5aa3f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:39.614482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:39.635717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7abea-85a5-4862-b368-27e2ea657096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:39.637100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7abea-85a5-4862-b368-27e2ea657096 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:39.637384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7abea-85a5-4862-b368-27e2ea657096 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:39.637659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7abea-85a5-4862-b368-27e2ea657096 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:39.638909 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7abea-85a5-4862-b368-27e2ea657096 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:39.640353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7abea-85a5-4862-b368-27e2ea657096 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:39.642467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 193/393] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:39 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:39.648405 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3c15d-a883-44ca-b262-295f8faab04e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:39.749205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:39.787466 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3c15d-a883-44ca-b262-295f8faab04e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:39.827398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33412ffb-92d1-460e-8670-0ada5bced8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:39.828294 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33412ffb-92d1-460e-8670-0ada5bced8f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:39.828574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33412ffb-92d1-460e-8670-0ada5bced8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:39.828848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33412ffb-92d1-460e-8670-0ada5bced8f5 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1/groups/0a2bd866b81249e7b752252ab4af5335/roles/950510c081de4ab6b12f383c0748450d` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:39.831115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33412ffb-92d1-460e-8670-0ada5bced8f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e080e6f49f52471c81ff5d29b80d3bb1, group_id=0a2bd866b81249e7b752252ab4af5335, role_id=950510c081de4ab6b12f383c0748450d)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:39.832550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33412ffb-92d1-460e-8670-0ada5bced8f5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:39.854035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 201/394] 10.222.0.60 () {58 vars in 1442 bytes} [Mon Jul 25 12:26:39 2022] PUT /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1/groups/0a2bd866b81249e7b752252ab4af5335/roles/950510c081de4ab6b12f383c0748450d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:40.356870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e1f62-a0de-46ef-9336-2344a8535025 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:40.357396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e1f62-a0de-46ef-9336-2344a8535025 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:40.357396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e1f62-a0de-46ef-9336-2344a8535025 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:40.359094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 194/395] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:26:40.364936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59904530-a096-4ee0-860a-0ee9d64cc4a3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:40.365229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59904530-a096-4ee0-860a-0ee9d64cc4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:40.365509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59904530-a096-4ee0-860a-0ee9d64cc4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:40.383838 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-59904530-a096-4ee0-860a-0ee9d64cc4a3 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:40.385037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-59904530-a096-4ee0-860a-0ee9d64cc4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:40.468299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 202/396] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:40.502860 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e1f62-a0de-46ef-9336-2344a8535025 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:40.528705 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112e367-f8e4-4fb9-a990-1df26249a61a None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:40.529005 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112e367-f8e4-4fb9-a990-1df26249a61a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:40.529288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112e367-f8e4-4fb9-a990-1df26249a61a None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:40.530923 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 203/397] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:26:40.535967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773146ad-6998-4f90-9473-85a736fb5aa3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:40.536255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773146ad-6998-4f90-9473-85a736fb5aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:40.536534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773146ad-6998-4f90-9473-85a736fb5aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:40.565158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-773146ad-6998-4f90-9473-85a736fb5aa3 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:40.566751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-773146ad-6998-4f90-9473-85a736fb5aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:40.646659 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 204/398] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:40.647365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:40.670793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773146ad-6998-4f90-9473-85a736fb5aa3 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:40.726688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e1f62-a0de-46ef-9336-2344a8535025 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:40.791882 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773146ad-6998-4f90-9473-85a736fb5aa3 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:40.833499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe4516-0cd5-4166-944a-61736799b902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:40.834508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe4516-0cd5-4166-944a-61736799b902 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:40.834895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe4516-0cd5-4166-944a-61736799b902 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:40.835279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe4516-0cd5-4166-944a-61736799b902 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:40.836662 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f3159-1611-4dad-903d-46e15fe3adf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:40.837483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f3159-1611-4dad-903d-46e15fe3adf6 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:40.837770 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f3159-1611-4dad-903d-46e15fe3adf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:40.838079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f3159-1611-4dad-903d-46e15fe3adf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:40.847801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe4516-0cd5-4166-944a-61736799b902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-d1fe4516-0cd5-4166-944a-61736799b902 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:40.860711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:40.861670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:40.862527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 205/399] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:26:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:26:40.868492 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe4516-0cd5-4166-944a-61736799b902 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:40.874526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f3159-1611-4dad-903d-46e15fe3adf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-cb4f3159-1611-4dad-903d-46e15fe3adf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:40.886786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:40.887683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:40.888464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 195/400] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:40.898904 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f3159-1611-4dad-903d-46e15fe3adf6 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:40.938316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:40.982077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe4516-0cd5-4166-944a-61736799b902 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:41.019896 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ef4a8-2db3-4fda-a736-e732752186c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:41.020602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ef4a8-2db3-4fda-a736-e732752186c9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:41.020887 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ef4a8-2db3-4fda-a736-e732752186c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:41.021163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ef4a8-2db3-4fda-a736-e732752186c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:41.021958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ef4a8-2db3-4fda-a736-e732752186c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:41.022246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ef4a8-2db3-4fda-a736-e732752186c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:41.023631 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ef4a8-2db3-4fda-a736-e732752186c9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:41.024300 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:41.035595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 206/401] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:41.043375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ef4a8-2db3-4fda-a736-e732752186c9 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:41.114962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f3159-1611-4dad-903d-46e15fe3adf6 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:41.190406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02584df1-79d3-4818-9643-b9d17d58da35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:41.193681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02584df1-79d3-4818-9643-b9d17d58da35 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:41.195015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02584df1-79d3-4818-9643-b9d17d58da35 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:41.198623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02584df1-79d3-4818-9643-b9d17d58da35 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:41.200664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02584df1-79d3-4818-9643-b9d17d58da35 None admin] RBAC: Adding query filter params (name=member) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:41.201363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02584df1-79d3-4818-9643-b9d17d58da35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:41.204173 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02584df1-79d3-4818-9643-b9d17d58da35 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:41.221720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 196/402] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:41.224023 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ef4a8-2db3-4fda-a736-e732752186c9 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:41.227298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02584df1-79d3-4818-9643-b9d17d58da35 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:41.252879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:41.302158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b13948-a826-453b-a7cd-a5ccdbe47df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:41.303020 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b13948-a826-453b-a7cd-a5ccdbe47df9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:41.303298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b13948-a826-453b-a7cd-a5ccdbe47df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:41.303570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b13948-a826-453b-a7cd-a5ccdbe47df9 None admin] PATH_INFO: `/v3/groups/0a2bd866b81249e7b752252ab4af5335` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:41.309341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b13948-a826-453b-a7cd-a5ccdbe47df9 None admin] RBAC: Authorizing `identity:get_group(group_id=0a2bd866b81249e7b752252ab4af5335)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:41.310813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b13948-a826-453b-a7cd-a5ccdbe47df9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:41.312757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 207/403] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:41 2022] GET /identity/v3/groups/0a2bd866b81249e7b752252ab4af5335 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:41.318004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b13948-a826-453b-a7cd-a5ccdbe47df9 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:41.337425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:41.345409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02584df1-79d3-4818-9643-b9d17d58da35 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:41.386202 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37e919-00f7-4407-998c-29c284f5f5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:41.387821 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37e919-00f7-4407-998c-29c284f5f5af None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:41.388255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37e919-00f7-4407-998c-29c284f5f5af None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:41.388775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37e919-00f7-4407-998c-29c284f5f5af None admin] PATH_INFO: `/v3/groups/0a2bd866b81249e7b752252ab4af5335` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:41.390165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b37e919-00f7-4407-998c-29c284f5f5af None admin] RBAC: Authorizing `identity:get_group(group_id=0a2bd866b81249e7b752252ab4af5335)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:41.391942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b37e919-00f7-4407-998c-29c284f5f5af None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:41.393849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 197/404] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:41 2022] GET /identity/v3/groups/0a2bd866b81249e7b752252ab4af5335 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:41.395402 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080c909-c0c3-43b7-8664-3fd7ca951654 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:41.395900 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080c909-c0c3-43b7-8664-3fd7ca951654 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:41.396389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080c909-c0c3-43b7-8664-3fd7ca951654 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:41.398089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 198/405] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:41.399152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080c909-c0c3-43b7-8664-3fd7ca951654 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:41.442441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b13948-a826-453b-a7cd-a5ccdbe47df9 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:41.480310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be935c-7956-404a-85a1-6013e989f3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:41.481724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be935c-7956-404a-85a1-6013e989f3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:41.481854 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be935c-7956-404a-85a1-6013e989f3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:41.482083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be935c-7956-404a-85a1-6013e989f3f4 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:41.483330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be935c-7956-404a-85a1-6013e989f3f4 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:41.484774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be935c-7956-404a-85a1-6013e989f3f4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:41.486859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 208/406] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:41 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:41.488197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef21f7-934f-4a34-b6ff-76e5c79fdd33 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:41.488493 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef21f7-934f-4a34-b6ff-76e5c79fdd33 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:41.488756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef21f7-934f-4a34-b6ff-76e5c79fdd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:41.508044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-81ef21f7-934f-4a34-b6ff-76e5c79fdd33 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:41.509265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-81ef21f7-934f-4a34-b6ff-76e5c79fdd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:41.516752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080c909-c0c3-43b7-8664-3fd7ca951654 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:41.554916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2091dab-df5a-44bf-88ad-26fd6739d580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:41.556174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2091dab-df5a-44bf-88ad-26fd6739d580 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:41.556664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2091dab-df5a-44bf-88ad-26fd6739d580 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:41.557179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2091dab-df5a-44bf-88ad-26fd6739d580 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:41.558708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2091dab-df5a-44bf-88ad-26fd6739d580 None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:41.560388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2091dab-df5a-44bf-88ad-26fd6739d580 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:41.562283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:41.562969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 199/407] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:41 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:41.563994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2091dab-df5a-44bf-88ad-26fd6739d580 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:41.585394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 209/408] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:41.586407 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef21f7-934f-4a34-b6ff-76e5c79fdd33 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:41.625635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:41.707170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef21f7-934f-4a34-b6ff-76e5c79fdd33 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:41.721169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2091dab-df5a-44bf-88ad-26fd6739d580 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:41.747560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4ca83-9c54-4775-8d78-a9e5fbafe970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:41.748718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4ca83-9c54-4775-8d78-a9e5fbafe970 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:41.749159 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4ca83-9c54-4775-8d78-a9e5fbafe970 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:41.749496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4ca83-9c54-4775-8d78-a9e5fbafe970 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e/groups/0a2bd866b81249e7b752252ab4af5335/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:41.752056 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4ca83-9c54-4775-8d78-a9e5fbafe970 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb4447b59c0746648ea2c2d04f5e3c7e, group_id=0a2bd866b81249e7b752252ab4af5335, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:41.754255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4ca83-9c54-4775-8d78-a9e5fbafe970 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:41.761254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40938-cf81-4939-bab8-5c3f5bb735a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:41.762232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40938-cf81-4939-bab8-5c3f5bb735a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:41.762523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40938-cf81-4939-bab8-5c3f5bb735a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:41.762847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40938-cf81-4939-bab8-5c3f5bb735a7 None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e/groups/0a2bd866b81249e7b752252ab4af5335/roles/950510c081de4ab6b12f383c0748450d` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:41.765387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b40938-cf81-4939-bab8-5c3f5bb735a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb4447b59c0746648ea2c2d04f5e3c7e, group_id=0a2bd866b81249e7b752252ab4af5335, role_id=950510c081de4ab6b12f383c0748450d)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:41.767110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b40938-cf81-4939-bab8-5c3f5bb735a7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:41.771848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 210/409] 10.222.0.60 () {58 vars in 1442 bytes} [Mon Jul 25 12:26:41 2022] PUT /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e/groups/0a2bd866b81249e7b752252ab4af5335/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:41.772995 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4ca83-9c54-4775-8d78-a9e5fbafe970 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:41.800762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 200/410] 10.222.0.60 () {58 vars in 1442 bytes} [Mon Jul 25 12:26:41 2022] PUT /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e/groups/0a2bd866b81249e7b752252ab4af5335/roles/950510c081de4ab6b12f383c0748450d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:41.824991 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:41.897305 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4ca83-9c54-4775-8d78-a9e5fbafe970 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:41.936264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8638ee-d25e-4fa5-b4bf-e3efcd35f929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:41.937396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8638ee-d25e-4fa5-b4bf-e3efcd35f929 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:41.937900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8638ee-d25e-4fa5-b4bf-e3efcd35f929 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:41.938448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8638ee-d25e-4fa5-b4bf-e3efcd35f929 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:41.951923 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8638ee-d25e-4fa5-b4bf-e3efcd35f929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-8d8638ee-d25e-4fa5-b4bf-e3efcd35f929 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:41.960416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:41.961376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:41.962209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:41.962209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:41.962209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:41.962209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:41.962209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:41.962209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:41.962209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:41.962209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:41.962209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:41.962209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:41.962209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:41.962209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:41.964996 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 211/411] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:26:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:41.971206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40938-cf81-4939-bab8-5c3f5bb735a7 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:42.091630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:42.103395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40938-cf81-4939-bab8-5c3f5bb735a7 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:42.140900 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a043d7-80cf-4175-9cfa-1b9c8ec5dc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:42.141974 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a043d7-80cf-4175-9cfa-1b9c8ec5dc84 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:42.142486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a043d7-80cf-4175-9cfa-1b9c8ec5dc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:42.143150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a043d7-80cf-4175-9cfa-1b9c8ec5dc84 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:42.144105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a043d7-80cf-4175-9cfa-1b9c8ec5dc84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:42.144614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a043d7-80cf-4175-9cfa-1b9c8ec5dc84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:42.146280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a043d7-80cf-4175-9cfa-1b9c8ec5dc84 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:42.156461 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 201/412] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:42.162413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8638ee-d25e-4fa5-b4bf-e3efcd35f929 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:42.239896 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb270cf-78f8-497f-b9e6-5899017a70e6 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:42.240740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb270cf-78f8-497f-b9e6-5899017a70e6 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:42.241693 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb270cf-78f8-497f-b9e6-5899017a70e6 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:42.244325 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 202/413] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:42.250675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81554752-4ade-4683-9161-eafbe88cc1d4 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:42.250960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81554752-4ade-4683-9161-eafbe88cc1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:42.251239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81554752-4ade-4683-9161-eafbe88cc1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:42.279152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-81554752-4ade-4683-9161-eafbe88cc1d4 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:42.284120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-81554752-4ade-4683-9161-eafbe88cc1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:42.346452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:42.349815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8638ee-d25e-4fa5-b4bf-e3efcd35f929 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:42.390638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 203/414] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:42.400430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000151d4-8cbb-41af-8515-9471f05ec2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:42.401321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000151d4-8cbb-41af-8515-9471f05ec2cc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:42.401600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000151d4-8cbb-41af-8515-9471f05ec2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:42.401879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000151d4-8cbb-41af-8515-9471f05ec2cc None admin] PATH_INFO: `/v3/groups/7b86e8287a634531a9027197ca3d0ac4` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:42.403079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000151d4-8cbb-41af-8515-9471f05ec2cc None admin] RBAC: Authorizing `identity:get_group(group_id=7b86e8287a634531a9027197ca3d0ac4)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:42.404529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000151d4-8cbb-41af-8515-9471f05ec2cc None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:42.406743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 212/415] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:42 2022] GET /identity/v3/groups/7b86e8287a634531a9027197ca3d0ac4 => generated 229 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:42.415009 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81554752-4ade-4683-9161-eafbe88cc1d4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:42.417424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000151d4-8cbb-41af-8515-9471f05ec2cc None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:42.563459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:42.595753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000151d4-8cbb-41af-8515-9471f05ec2cc None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:42.598173 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81554752-4ade-4683-9161-eafbe88cc1d4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:42.642804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a43f13-8f70-4994-97b7-8d9932136459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:42.643602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a43f13-8f70-4994-97b7-8d9932136459 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:42.643796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4d5a2-531e-4377-a843-cecd75c5e270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:42.643907 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a43f13-8f70-4994-97b7-8d9932136459 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:42.644143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a43f13-8f70-4994-97b7-8d9932136459 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:42.644611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4d5a2-531e-4377-a843-cecd75c5e270 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:42.644898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4d5a2-531e-4377-a843-cecd75c5e270 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:42.645173 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4d5a2-531e-4377-a843-cecd75c5e270 None admin] PATH_INFO: `/v3/projects/61157336119141239b389248fadfbdf0` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:42.646703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4d5a2-531e-4377-a843-cecd75c5e270 None admin] RBAC: Authorizing `identity:get_project(project_id=61157336119141239b389248fadfbdf0)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:42.648533 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4d5a2-531e-4377-a843-cecd75c5e270 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:42.650996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 204/416] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:42 2022] GET /identity/v3/projects/61157336119141239b389248fadfbdf0 => generated 344 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:42.652985 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624201b9-70f8-4bb3-a4d5-91b6fda74edc None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:42.653254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624201b9-70f8-4bb3-a4d5-91b6fda74edc None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:42.653531 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624201b9-70f8-4bb3-a4d5-91b6fda74edc None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:42.655123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a43f13-8f70-4994-97b7-8d9932136459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:42.659119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 205/417] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:42.659890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624201b9-70f8-4bb3-a4d5-91b6fda74edc None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-44a43f13-8f70-4994-97b7-8d9932136459 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:42.663403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:42.664299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:42.665098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 213/418] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:42.666337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33b605-493e-41b6-a499-69562c6e99f5 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:42.666337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33b605-493e-41b6-a499-69562c6e99f5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:42.666649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33b605-493e-41b6-a499-69562c6e99f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:42.686133 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0d33b605-493e-41b6-a499-69562c6e99f5 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:42.687318 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0d33b605-493e-41b6-a499-69562c6e99f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:42.716817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:42.786306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 214/419] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:42.787651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33b605-493e-41b6-a499-69562c6e99f5 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:42.789583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624201b9-70f8-4bb3-a4d5-91b6fda74edc None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:42.826429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa000b-231f-402a-acbf-785a7d915b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:42.827825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa000b-231f-402a-acbf-785a7d915b75 None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:42.828269 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa000b-231f-402a-acbf-785a7d915b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:42.828701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa000b-231f-402a-acbf-785a7d915b75 None admin] PATH_INFO: `/v3/projects/61157336119141239b389248fadfbdf0/groups/7b86e8287a634531a9027197ca3d0ac4/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:42.831081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa000b-231f-402a-acbf-785a7d915b75 None admin] RBAC: Authorizing `identity:create_grant(project_id=61157336119141239b389248fadfbdf0, group_id=7b86e8287a634531a9027197ca3d0ac4, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:42.833294 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa000b-231f-402a-acbf-785a7d915b75 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:42.849435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 206/420] 10.222.0.60 () {58 vars in 1442 bytes} [Mon Jul 25 12:26:42 2022] PUT /identity/v3/projects/61157336119141239b389248fadfbdf0/groups/7b86e8287a634531a9027197ca3d0ac4/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:26:42.850660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa000b-231f-402a-acbf-785a7d915b75 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:42.870863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:43.005954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33b605-493e-41b6-a499-69562c6e99f5 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:43.030664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa000b-231f-402a-acbf-785a7d915b75 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:43.043684 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cb9ee-d3c5-4be2-8ee1-1e50c003c6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:43.044352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cb9ee-d3c5-4be2-8ee1-1e50c003c6ed None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:43.044635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cb9ee-d3c5-4be2-8ee1-1e50c003c6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:43.044911 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cb9ee-d3c5-4be2-8ee1-1e50c003c6ed None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:43.045667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cb9ee-d3c5-4be2-8ee1-1e50c003c6ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:43.046005 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cb9ee-d3c5-4be2-8ee1-1e50c003c6ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:43.047386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cb9ee-d3c5-4be2-8ee1-1e50c003c6ed None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:43.061681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 215/421] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:43.062599 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cb9ee-d3c5-4be2-8ee1-1e50c003c6ed None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:43.070291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9fb86-b579-43f4-af8c-17d64cbd96d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:43.071483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9fb86-b579-43f4-af8c-17d64cbd96d8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:43.071981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9fb86-b579-43f4-af8c-17d64cbd96d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:43.072445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9fb86-b579-43f4-af8c-17d64cbd96d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:43.085199 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9fb86-b579-43f4-af8c-17d64cbd96d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:43.087516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-14c9fb86-b579-43f4-af8c-17d64cbd96d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:43.095015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:43.096012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:43.096791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:43.096791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:43.096791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:43.096791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:43.096791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:43.096791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:43.096791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:43.096791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:43.096791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:43.096791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:43.096791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:43.096791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:43.099668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 207/422] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:26:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:43.111996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9fb86-b579-43f4-af8c-17d64cbd96d8 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:43.181598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cb9ee-d3c5-4be2-8ee1-1e50c003c6ed None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:43.204534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:43.229602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1e393-45ed-4ce1-aca0-429724e516c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:43.231527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1e393-45ed-4ce1-aca0-429724e516c9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:43.231984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1e393-45ed-4ce1-aca0-429724e516c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:43.232464 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1e393-45ed-4ce1-aca0-429724e516c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:43.236497 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9fb86-b579-43f4-af8c-17d64cbd96d8 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:43.276935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cd07e-ec7a-49ae-9956-277238909942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:43.277760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cd07e-ec7a-49ae-9956-277238909942 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:43.278138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cd07e-ec7a-49ae-9956-277238909942 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:43.278494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cd07e-ec7a-49ae-9956-277238909942 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:43.279300 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cd07e-ec7a-49ae-9956-277238909942 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:43.279677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cd07e-ec7a-49ae-9956-277238909942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:43.281153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cd07e-ec7a-49ae-9956-277238909942 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:43.289818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 208/423] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:43.295379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cd07e-ec7a-49ae-9956-277238909942 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:43.322548 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1e393-45ed-4ce1-aca0-429724e516c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:43.325381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1e393-45ed-4ce1-aca0-429724e516c9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:43.348651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:43.418115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cd07e-ec7a-49ae-9956-277238909942 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:43.446892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 216/424] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:43.452376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1e393-45ed-4ce1-aca0-429724e516c9 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:43.473210 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbbe4c-c803-4e98-a705-8b8519119bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:43.473866 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbbe4c-c803-4e98-a705-8b8519119bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:43.474170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbbe4c-c803-4e98-a705-8b8519119bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:43.474445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbbe4c-c803-4e98-a705-8b8519119bf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:43.480335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:43.613439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1e393-45ed-4ce1-aca0-429724e516c9 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:43.616049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbbe4c-c803-4e98-a705-8b8519119bf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:43.617509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbbe4c-c803-4e98-a705-8b8519119bf8 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:43.638286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:43.675239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ab05a-25b0-4e46-a35d-cabe2fc962d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:43.676028 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ab05a-25b0-4e46-a35d-cabe2fc962d8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:43.676305 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ab05a-25b0-4e46-a35d-cabe2fc962d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:43.676575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ab05a-25b0-4e46-a35d-cabe2fc962d8 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:43.678631 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ab05a-25b0-4e46-a35d-cabe2fc962d8 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:43.680231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ab05a-25b0-4e46-a35d-cabe2fc962d8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:43.682087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 217/425] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:43 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:43.687822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ab05a-25b0-4e46-a35d-cabe2fc962d8 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:43.741897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 209/426] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:26:43.749002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbbe4c-c803-4e98-a705-8b8519119bf8 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:43.802348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:43.862644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbbe4c-c803-4e98-a705-8b8519119bf8 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:43.900935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396a738-6b99-4e2a-8a47-cfaef12360b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:43.902681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396a738-6b99-4e2a-8a47-cfaef12360b7 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:43.903144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396a738-6b99-4e2a-8a47-cfaef12360b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:43.903594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396a738-6b99-4e2a-8a47-cfaef12360b7 None admin] PATH_INFO: `/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:43.904954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396a738-6b99-4e2a-8a47-cfaef12360b7 None admin] RBAC: Authorizing `identity:get_project(project_id=e080e6f49f52471c81ff5d29b80d3bb1)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:43.907180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396a738-6b99-4e2a-8a47-cfaef12360b7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:43.909889 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 210/427] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:43 2022] GET /identity/v3/projects/e080e6f49f52471c81ff5d29b80d3bb1 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:43.916891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396a738-6b99-4e2a-8a47-cfaef12360b7 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:43.964688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ab05a-25b0-4e46-a35d-cabe2fc962d8 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:44.053756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64b1de-5302-4f5a-8316-1cb6506bb225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:44.055016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64b1de-5302-4f5a-8316-1cb6506bb225 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:44.055147 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64b1de-5302-4f5a-8316-1cb6506bb225 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:44.055397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64b1de-5302-4f5a-8316-1cb6506bb225 None admin] PATH_INFO: `/v3/groups/0a2bd866b81249e7b752252ab4af5335` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:44.056747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64b1de-5302-4f5a-8316-1cb6506bb225 None admin] RBAC: Authorizing `identity:get_group(group_id=0a2bd866b81249e7b752252ab4af5335)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:44.058617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64b1de-5302-4f5a-8316-1cb6506bb225 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:44.065156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 218/428] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:43 2022] GET /identity/v3/groups/0a2bd866b81249e7b752252ab4af5335 => generated 226 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:44.066314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64b1de-5302-4f5a-8316-1cb6506bb225 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:44.067054 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396a738-6b99-4e2a-8a47-cfaef12360b7 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:44.070215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:44.125391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:44.139003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9c216-0c0f-4e12-b731-e4f8ecbf6f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:44.140106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9c216-0c0f-4e12-b731-e4f8ecbf6f25 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:44.140388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9c216-0c0f-4e12-b731-e4f8ecbf6f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:44.141523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9c216-0c0f-4e12-b731-e4f8ecbf6f25 None admin] PATH_INFO: `/v3/groups/0a2bd866b81249e7b752252ab4af5335` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:44.142761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9c216-0c0f-4e12-b731-e4f8ecbf6f25 None admin] RBAC: Authorizing `identity:get_group(group_id=0a2bd866b81249e7b752252ab4af5335)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:44.144203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9c216-0c0f-4e12-b731-e4f8ecbf6f25 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:44.146635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 211/429] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:43 2022] GET /identity/v3/groups/0a2bd866b81249e7b752252ab4af5335 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:44.148267 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b45021-1325-41da-97d3-94a95de39d32 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:44.148559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b45021-1325-41da-97d3-94a95de39d32 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:44.148832 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b45021-1325-41da-97d3-94a95de39d32 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:44.150065 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 212/430] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:44.153946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b45021-1325-41da-97d3-94a95de39d32 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:44.189032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64b1de-5302-4f5a-8316-1cb6506bb225 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:44.226195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88543e09-e940-4ea8-b84d-d73ce3e0369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:44.227063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88543e09-e940-4ea8-b84d-d73ce3e0369c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:44.227352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88543e09-e940-4ea8-b84d-d73ce3e0369c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:44.227626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88543e09-e940-4ea8-b84d-d73ce3e0369c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:44.228454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88543e09-e940-4ea8-b84d-d73ce3e0369c None admin] RBAC: Adding query filter params (group.id=0a2bd866b81249e7b752252ab4af5335, role.id=86e07561a0854fc28db2e9f357e53d66, scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:44.228807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88543e09-e940-4ea8-b84d-d73ce3e0369c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:44.230324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88543e09-e940-4ea8-b84d-d73ce3e0369c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:44.237069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 219/431] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Jul 25 12:26:44 2022] GET /identity/v3/role_assignments?group.id=0a2bd866b81249e7b752252ab4af5335&scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1&role.id=86e07561a0854fc28db2e9f357e53d66 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:44.238340 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199fe7b-ecd9-407e-babd-6d34aeb41aab None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:44.238624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199fe7b-ecd9-407e-babd-6d34aeb41aab None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:44.238894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199fe7b-ecd9-407e-babd-6d34aeb41aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:44.259445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6199fe7b-ecd9-407e-babd-6d34aeb41aab None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:44.261241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6199fe7b-ecd9-407e-babd-6d34aeb41aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:44.298723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b45021-1325-41da-97d3-94a95de39d32 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:44.338037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 220/432] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:44.358927 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06fa44-58ce-4baf-84a6-816175b6d528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:44.359593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06fa44-58ce-4baf-84a6-816175b6d528 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:44.359878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06fa44-58ce-4baf-84a6-816175b6d528 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:44.360153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06fa44-58ce-4baf-84a6-816175b6d528 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:44.360991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06fa44-58ce-4baf-84a6-816175b6d528 None admin] RBAC: Adding query filter params (group.id=0a2bd866b81249e7b752252ab4af5335, role.id=950510c081de4ab6b12f383c0748450d, scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:44.361304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06fa44-58ce-4baf-84a6-816175b6d528 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:44.363076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06fa44-58ce-4baf-84a6-816175b6d528 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:44.363394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199fe7b-ecd9-407e-babd-6d34aeb41aab None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:44.369374 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 213/433] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Jul 25 12:26:44 2022] GET /identity/v3/role_assignments?group.id=0a2bd866b81249e7b752252ab4af5335&scope.project.id=e080e6f49f52471c81ff5d29b80d3bb1&role.id=950510c081de4ab6b12f383c0748450d => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:44.421950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:44.481940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199fe7b-ecd9-407e-babd-6d34aeb41aab None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:44.520886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e87baa-2f4e-48f2-b6fb-b3002c3f311f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:44.521707 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e87baa-2f4e-48f2-b6fb-b3002c3f311f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:44.522016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e87baa-2f4e-48f2-b6fb-b3002c3f311f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:44.522293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e87baa-2f4e-48f2-b6fb-b3002c3f311f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:44.532153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cdf8d-9ab2-4292-8053-92f369f474cb None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:44.532777 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cdf8d-9ab2-4292-8053-92f369f474cb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:44.532777 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cdf8d-9ab2-4292-8053-92f369f474cb None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:44.535088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e87baa-2f4e-48f2-b6fb-b3002c3f311f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:44.535596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 214/434] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:44.540017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5fdd9-6928-42a3-8cec-fc5ef7c25105 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:44.540307 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5fdd9-6928-42a3-8cec-fc5ef7c25105 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:44.540589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5fdd9-6928-42a3-8cec-fc5ef7c25105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-d3e87baa-2f4e-48f2-b6fb-b3002c3f311f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:44.543223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:44.545316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:44.547321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 221/435] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:26:44.551439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e87baa-2f4e-48f2-b6fb-b3002c3f311f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:44.584201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-ecf5fdd9-6928-42a3-8cec-fc5ef7c25105 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:44.585411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ecf5fdd9-6928-42a3-8cec-fc5ef7c25105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:44.635440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:44.672135 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e87baa-2f4e-48f2-b6fb-b3002c3f311f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:44.684552 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:44.706018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 215/436] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:44.726680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6b05c-d6ca-4f8d-9a11-ff4254a88ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:44.726897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5fdd9-6928-42a3-8cec-fc5ef7c25105 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:44.727830 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6b05c-d6ca-4f8d-9a11-ff4254a88ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:44.728113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6b05c-d6ca-4f8d-9a11-ff4254a88ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:44.728387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6b05c-d6ca-4f8d-9a11-ff4254a88ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:44.729396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6b05c-d6ca-4f8d-9a11-ff4254a88ae4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:44.730308 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6b05c-d6ca-4f8d-9a11-ff4254a88ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:44.732004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6b05c-d6ca-4f8d-9a11-ff4254a88ae4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:44.745507 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 222/437] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:26:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:44.751121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6b05c-d6ca-4f8d-9a11-ff4254a88ae4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:44.857433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5fdd9-6928-42a3-8cec-fc5ef7c25105 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:44.883778 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6b05c-d6ca-4f8d-9a11-ff4254a88ae4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:44.902857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94c640-a2f5-4230-b844-08c94606929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:44.903932 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94c640-a2f5-4230-b844-08c94606929d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:44.904507 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94c640-a2f5-4230-b844-08c94606929d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:44.906302 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94c640-a2f5-4230-b844-08c94606929d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:44.924987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94c640-a2f5-4230-b844-08c94606929d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:44.929212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89cc0e-52ba-44a1-9e84-26cf68c1a795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:44.929860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89cc0e-52ba-44a1-9e84-26cf68c1a795 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:44.930171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89cc0e-52ba-44a1-9e84-26cf68c1a795 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:44.930447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89cc0e-52ba-44a1-9e84-26cf68c1a795 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-ee94c640-a2f5-4230-b844-08c94606929d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:44.934637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:44.935541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:44.936354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:44.936354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:44.936354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:44.936354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:44.936354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:44.936354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:44.936354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:44.936354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:44.936354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:44.936354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:44.936354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:44.936354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:44.937093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 216/438] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:26:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:26:44.943109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94c640-a2f5-4230-b844-08c94606929d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:44.981181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:44.984565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:45.025447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89cc0e-52ba-44a1-9e84-26cf68c1a795 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:45.027831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89cc0e-52ba-44a1-9e84-26cf68c1a795 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:45.085025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94c640-a2f5-4230-b844-08c94606929d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:45.136458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac61f65-01f7-41a3-b18c-a59886feec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:45.137747 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac61f65-01f7-41a3-b18c-a59886feec32 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:45.138671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac61f65-01f7-41a3-b18c-a59886feec32 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:45.139139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac61f65-01f7-41a3-b18c-a59886feec32 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:45.140366 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac61f65-01f7-41a3-b18c-a59886feec32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:45.140679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac61f65-01f7-41a3-b18c-a59886feec32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:45.142187 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac61f65-01f7-41a3-b18c-a59886feec32 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:45.152586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 217/439] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:45.152764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 223/440] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:26:45.158191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac61f65-01f7-41a3-b18c-a59886feec32 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:45.158336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89cc0e-52ba-44a1-9e84-26cf68c1a795 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:45.243807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:45.260151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:45.273892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89cc0e-52ba-44a1-9e84-26cf68c1a795 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:45.275459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac61f65-01f7-41a3-b18c-a59886feec32 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:45.307509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd3021-8205-49b9-a1d4-9e6faf09a52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:45.308370 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd3021-8205-49b9-a1d4-9e6faf09a52e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:45.308655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd3021-8205-49b9-a1d4-9e6faf09a52e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:45.308864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e704a56-1227-4bea-9072-6df6c4df88bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:45.308973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd3021-8205-49b9-a1d4-9e6faf09a52e None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:45.309504 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e704a56-1227-4bea-9072-6df6c4df88bb None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:45.309784 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e704a56-1227-4bea-9072-6df6c4df88bb None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:45.310086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e704a56-1227-4bea-9072-6df6c4df88bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:45.310181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd3021-8205-49b9-a1d4-9e6faf09a52e None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:45.311561 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd3021-8205-49b9-a1d4-9e6faf09a52e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:45.313255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 224/441] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:45 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:45.314598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3e8d1-25a9-449b-9d70-39e229524f5b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:45.314890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3e8d1-25a9-449b-9d70-39e229524f5b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:45.315171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3e8d1-25a9-449b-9d70-39e229524f5b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:45.316392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 225/442] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:45.318553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3e8d1-25a9-449b-9d70-39e229524f5b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:45.377860 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e704a56-1227-4bea-9072-6df6c4df88bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:45.379338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e704a56-1227-4bea-9072-6df6c4df88bb None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:45.418537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3e8d1-25a9-449b-9d70-39e229524f5b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:45.451077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a412400-db3d-49e4-b4a7-4838770b3ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:45.451981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a412400-db3d-49e4-b4a7-4838770b3ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:45.452270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a412400-db3d-49e4-b4a7-4838770b3ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:45.452545 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a412400-db3d-49e4-b4a7-4838770b3ed2 None admin] PATH_INFO: `/v3/groups/0a2bd866b81249e7b752252ab4af5335` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:45.453703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a412400-db3d-49e4-b4a7-4838770b3ed2 None admin] RBAC: Authorizing `identity:get_group(group_id=0a2bd866b81249e7b752252ab4af5335)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:45.455150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a412400-db3d-49e4-b4a7-4838770b3ed2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:45.456756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 226/443] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:45 2022] GET /identity/v3/groups/0a2bd866b81249e7b752252ab4af5335 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:45.457857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9978ef-d9dd-4eb0-8f98-a37a28138084 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:45.458164 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9978ef-d9dd-4eb0-8f98-a37a28138084 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:45.458440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9978ef-d9dd-4eb0-8f98-a37a28138084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:45.461377 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:45.476607 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2b9978ef-d9dd-4eb0-8f98-a37a28138084 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:45.477797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2b9978ef-d9dd-4eb0-8f98-a37a28138084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:45.478650 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 218/444] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:45.479933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e704a56-1227-4bea-9072-6df6c4df88bb None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:45.480546 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:45.544969 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 227/445] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:45.545617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9978ef-d9dd-4eb0-8f98-a37a28138084 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:45.598170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e704a56-1227-4bea-9072-6df6c4df88bb None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:45.634076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486203d2-6583-4969-8156-7c1d5eb9206f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:45.634721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486203d2-6583-4969-8156-7c1d5eb9206f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:45.634997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486203d2-6583-4969-8156-7c1d5eb9206f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:45.635271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486203d2-6583-4969-8156-7c1d5eb9206f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:45.636646 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486203d2-6583-4969-8156-7c1d5eb9206f None admin] RBAC: Adding query filter params (group.id=0a2bd866b81249e7b752252ab4af5335, role.id=86e07561a0854fc28db2e9f357e53d66, scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:45.636950 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486203d2-6583-4969-8156-7c1d5eb9206f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:45.638407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486203d2-6583-4969-8156-7c1d5eb9206f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:45.644289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 219/446] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Jul 25 12:26:45 2022] GET /identity/v3/role_assignments?group.id=0a2bd866b81249e7b752252ab4af5335&scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e&role.id=86e07561a0854fc28db2e9f357e53d66 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:45.644769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9978ef-d9dd-4eb0-8f98-a37a28138084 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:45.645031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486203d2-6583-4969-8156-7c1d5eb9206f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:45.688856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981d40e-517e-453d-a5cd-7e9f4ebdabee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:45.689741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981d40e-517e-453d-a5cd-7e9f4ebdabee None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:45.690068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981d40e-517e-453d-a5cd-7e9f4ebdabee None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:45.690344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981d40e-517e-453d-a5cd-7e9f4ebdabee None admin] PATH_INFO: `/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:45.691623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981d40e-517e-453d-a5cd-7e9f4ebdabee None admin] RBAC: Authorizing `identity:get_project(project_id=eb4447b59c0746648ea2c2d04f5e3c7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:45.693105 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981d40e-517e-453d-a5cd-7e9f4ebdabee None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:45.694977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 228/447] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:45 2022] GET /identity/v3/projects/eb4447b59c0746648ea2c2d04f5e3c7e => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:45.701252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981d40e-517e-453d-a5cd-7e9f4ebdabee None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:45.708899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:45.711149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:45.766494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486203d2-6583-4969-8156-7c1d5eb9206f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:45.802815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f92c45-4dd0-402b-9de7-31f659c28952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:45.804025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f92c45-4dd0-402b-9de7-31f659c28952 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:45.804025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f92c45-4dd0-402b-9de7-31f659c28952 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:45.804188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f92c45-4dd0-402b-9de7-31f659c28952 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:45.815342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f92c45-4dd0-402b-9de7-31f659c28952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:45.818374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981d40e-517e-453d-a5cd-7e9f4ebdabee None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-42f92c45-4dd0-402b-9de7-31f659c28952 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:45.824435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:45.825316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:45.826138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 220/448] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:26:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:45.833946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f92c45-4dd0-402b-9de7-31f659c28952 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:45.857578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a81aa4-c23b-42e7-8608-91b904b2cd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:45.858470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a81aa4-c23b-42e7-8608-91b904b2cd98 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:45.858757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a81aa4-c23b-42e7-8608-91b904b2cd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:45.859032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a81aa4-c23b-42e7-8608-91b904b2cd98 None admin] PATH_INFO: `/v3/groups/0a2bd866b81249e7b752252ab4af5335` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:45.861168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a81aa4-c23b-42e7-8608-91b904b2cd98 None admin] RBAC: Authorizing `identity:get_group(group_id=0a2bd866b81249e7b752252ab4af5335)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:45.862655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a81aa4-c23b-42e7-8608-91b904b2cd98 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:45.864587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 229/449] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:45 2022] GET /identity/v3/groups/0a2bd866b81249e7b752252ab4af5335 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:45.870541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a81aa4-c23b-42e7-8608-91b904b2cd98 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:45.954981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:45.958045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f92c45-4dd0-402b-9de7-31f659c28952 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:45.972240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:45.992178 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a81aa4-c23b-42e7-8608-91b904b2cd98 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:45.998093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017521e2-e517-4a50-8776-998e19a51909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:45.999040 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017521e2-e517-4a50-8776-998e19a51909 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:45.999463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017521e2-e517-4a50-8776-998e19a51909 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:45.999885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017521e2-e517-4a50-8776-998e19a51909 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:46.000857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017521e2-e517-4a50-8776-998e19a51909 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:46.001171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017521e2-e517-4a50-8776-998e19a51909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:46.002595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017521e2-e517-4a50-8776-998e19a51909 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:46.013778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 221/450] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:46.019458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017521e2-e517-4a50-8776-998e19a51909 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:46.029763 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95ea8e-76bc-4a14-a03f-a28cc512c99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:46.030450 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95ea8e-76bc-4a14-a03f-a28cc512c99e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:46.030726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95ea8e-76bc-4a14-a03f-a28cc512c99e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:46.030998 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95ea8e-76bc-4a14-a03f-a28cc512c99e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:46.031823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95ea8e-76bc-4a14-a03f-a28cc512c99e None admin] RBAC: Adding query filter params (group.id=0a2bd866b81249e7b752252ab4af5335, role.id=950510c081de4ab6b12f383c0748450d, scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:46.032129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95ea8e-76bc-4a14-a03f-a28cc512c99e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:46.033545 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95ea8e-76bc-4a14-a03f-a28cc512c99e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:46.039177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 230/451] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Jul 25 12:26:45 2022] GET /identity/v3/role_assignments?group.id=0a2bd866b81249e7b752252ab4af5335&scope.project.id=eb4447b59c0746648ea2c2d04f5e3c7e&role.id=950510c081de4ab6b12f383c0748450d => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:46.124034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017521e2-e517-4a50-8776-998e19a51909 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:46.158531 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65f926-ef81-4103-bc5f-7acd43965205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:46.159192 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65f926-ef81-4103-bc5f-7acd43965205 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:46.159478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65f926-ef81-4103-bc5f-7acd43965205 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:46.159754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65f926-ef81-4103-bc5f-7acd43965205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:46.204671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:46.231427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65f926-ef81-4103-bc5f-7acd43965205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:46.232879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65f926-ef81-4103-bc5f-7acd43965205 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:46.330826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 222/452] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:26:46.336581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95ea8e-76bc-4a14-a03f-a28cc512c99e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:46.434613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95ea8e-76bc-4a14-a03f-a28cc512c99e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:46.467063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e41e82-a3eb-4ecf-8543-43539414accd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:46.467924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e41e82-a3eb-4ecf-8543-43539414accd None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:46.468200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e41e82-a3eb-4ecf-8543-43539414accd None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:46.468467 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e41e82-a3eb-4ecf-8543-43539414accd None admin] PATH_INFO: `/v3/projects/61157336119141239b389248fadfbdf0` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:46.469630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e41e82-a3eb-4ecf-8543-43539414accd None admin] RBAC: Authorizing `identity:get_project(project_id=61157336119141239b389248fadfbdf0)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:46.471079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e41e82-a3eb-4ecf-8543-43539414accd None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:46.472745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 231/453] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:26:46 2022] GET /identity/v3/projects/61157336119141239b389248fadfbdf0 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:46.478266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65f926-ef81-4103-bc5f-7acd43965205 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:46.544482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:46.573839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65f926-ef81-4103-bc5f-7acd43965205 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:46.607439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94d2c3-d348-4227-b734-5557988ceb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:46.608258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94d2c3-d348-4227-b734-5557988ceb4e None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:46.608543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94d2c3-d348-4227-b734-5557988ceb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:46.608819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94d2c3-d348-4227-b734-5557988ceb4e None admin] PATH_INFO: `/v3/groups/7b86e8287a634531a9027197ca3d0ac4` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:46.610006 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94d2c3-d348-4227-b734-5557988ceb4e None admin] RBAC: Authorizing `identity:get_group(group_id=7b86e8287a634531a9027197ca3d0ac4)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:46.611441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94d2c3-d348-4227-b734-5557988ceb4e None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:46.613036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 223/454] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Jul 25 12:26:46 2022] GET /identity/v3/groups/7b86e8287a634531a9027197ca3d0ac4 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:46.618651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e41e82-a3eb-4ecf-8543-43539414accd None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:46.628290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:46.715373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e41e82-a3eb-4ecf-8543-43539414accd None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:46.747996 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e393892-5c8f-456e-b305-c650504dc53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:46.748636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e393892-5c8f-456e-b305-c650504dc53d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:46.748934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e393892-5c8f-456e-b305-c650504dc53d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:46.749197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e393892-5c8f-456e-b305-c650504dc53d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:46.750045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e393892-5c8f-456e-b305-c650504dc53d None admin] RBAC: Adding query filter params (group.id=7b86e8287a634531a9027197ca3d0ac4, role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=61157336119141239b389248fadfbdf0) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:46.750358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e393892-5c8f-456e-b305-c650504dc53d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:46.751782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e393892-5c8f-456e-b305-c650504dc53d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:46.757319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 232/455] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Jul 25 12:26:46 2022] GET /identity/v3/role_assignments?group.id=7b86e8287a634531a9027197ca3d0ac4&scope.project.id=61157336119141239b389248fadfbdf0&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:49.180472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e164c-5aab-4c2e-b15f-4b39977e36c7 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:49.181011 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e164c-5aab-4c2e-b15f-4b39977e36c7 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:49.181011 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e164c-5aab-4c2e-b15f-4b39977e36c7 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:49.184838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 224/456] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:49.188626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2676e5-f183-4a0a-b3ac-b798d594b5e1 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:49.189024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2676e5-f183-4a0a-b3ac-b798d594b5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:49.189185 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2676e5-f183-4a0a-b3ac-b798d594b5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:49.215497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7a2676e5-f183-4a0a-b3ac-b798d594b5e1 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:49.216702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7a2676e5-f183-4a0a-b3ac-b798d594b5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:49.283943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:49.288732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 233/457] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:49.336575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e164c-5aab-4c2e-b15f-4b39977e36c7 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:49.462667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e164c-5aab-4c2e-b15f-4b39977e36c7 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:49.476104 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1af474-06a2-47b7-8450-ba7307d34409 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:49.476687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1af474-06a2-47b7-8450-ba7307d34409 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:49.477165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1af474-06a2-47b7-8450-ba7307d34409 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:49.478971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 234/458] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:49.484932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f11724-c06b-478d-97be-94a808722d89 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:49.485447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f11724-c06b-478d-97be-94a808722d89 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:49.485961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f11724-c06b-478d-97be-94a808722d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:49.513624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc818e97-b660-4108-8ba6-9ceedc631f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:49.514848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc818e97-b660-4108-8ba6-9ceedc631f92 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:49.515329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc818e97-b660-4108-8ba6-9ceedc631f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:49.515799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc818e97-b660-4108-8ba6-9ceedc631f92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:49.517196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-23f11724-c06b-478d-97be-94a808722d89 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:49.518823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-23f11724-c06b-478d-97be-94a808722d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:49.533685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:49.538037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc818e97-b660-4108-8ba6-9ceedc631f92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:49.539490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc818e97-b660-4108-8ba6-9ceedc631f92 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-fc818e97-b660-4108-8ba6-9ceedc631f92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:49.564003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:49.564907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:49.565754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:49.565754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:49.565754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:49.565754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:49.565754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 225/459] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:49.581000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b446f5d-3e79-454b-ac23-111f55d21d1b None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:49.581580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b446f5d-3e79-454b-ac23-111f55d21d1b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:49.582057 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b446f5d-3e79-454b-ac23-111f55d21d1b None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:49.583826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 226/460] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:49.584779 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b446f5d-3e79-454b-ac23-111f55d21d1b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:49.619437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 235/461] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:49.621651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e457a-f1fe-418e-ba6f-5d4bbd031f4e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:49.622160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e457a-f1fe-418e-ba6f-5d4bbd031f4e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:49.622619 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e457a-f1fe-418e-ba6f-5d4bbd031f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:49.655802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d41e457a-f1fe-418e-ba6f-5d4bbd031f4e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:49.657609 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d41e457a-f1fe-418e-ba6f-5d4bbd031f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:49.724829 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b446f5d-3e79-454b-ac23-111f55d21d1b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:49.743327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 236/462] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:49.744524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e457a-f1fe-418e-ba6f-5d4bbd031f4e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:49.781395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b017d1-3ddf-47b1-84ba-2018d8e5889a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:49.782103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b017d1-3ddf-47b1-84ba-2018d8e5889a None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:49.782388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b017d1-3ddf-47b1-84ba-2018d8e5889a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:49.782669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b017d1-3ddf-47b1-84ba-2018d8e5889a None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:49.783759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b017d1-3ddf-47b1-84ba-2018d8e5889a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:49.784481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b017d1-3ddf-47b1-84ba-2018d8e5889a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:49.786032 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b017d1-3ddf-47b1-84ba-2018d8e5889a None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:49.810842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 227/463] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:49.812222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b017d1-3ddf-47b1-84ba-2018d8e5889a None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:49.814107 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:49.874590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e457a-f1fe-418e-ba6f-5d4bbd031f4e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:49.880915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:49.910937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9178d4-ac11-4eb1-93d7-add0fc9012e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:49.912007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9178d4-ac11-4eb1-93d7-add0fc9012e7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:49.912433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9178d4-ac11-4eb1-93d7-add0fc9012e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:49.912851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9178d4-ac11-4eb1-93d7-add0fc9012e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:49.924589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9178d4-ac11-4eb1-93d7-add0fc9012e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:49.926265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9178d4-ac11-4eb1-93d7-add0fc9012e7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:49.935983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b017d1-3ddf-47b1-84ba-2018d8e5889a None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-7b9178d4-ac11-4eb1-93d7-add0fc9012e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:49.944828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:49.945671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:49.946592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:49.946592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:49.946592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:49.946592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:49.950231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 237/464] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:49.951395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9178d4-ac11-4eb1-93d7-add0fc9012e7 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:49.972478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c704a-9de0-4d36-bca3-c0f9964ca811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:49.973279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c704a-9de0-4d36-bca3-c0f9964ca811 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:49.973563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c704a-9de0-4d36-bca3-c0f9964ca811 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:49.973838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c704a-9de0-4d36-bca3-c0f9964ca811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:49.984965 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c704a-9de0-4d36-bca3-c0f9964ca811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:49.986399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c704a-9de0-4d36-bca3-c0f9964ca811 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-c37c704a-9de0-4d36-bca3-c0f9964ca811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:50.004047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:50.004915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:50.005748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:50.005748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:50.005748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:50.005748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:50.005748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 228/465] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:50.006572 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5f0ce-6d42-44ec-8ba0-61b28e1d77c6 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:50.006864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5f0ce-6d42-44ec-8ba0-61b28e1d77c6 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:50.007144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5f0ce-6d42-44ec-8ba0-61b28e1d77c6 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:50.010665 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 229/466] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:50.010665 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5f0ce-6d42-44ec-8ba0-61b28e1d77c6 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:50.069218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9178d4-ac11-4eb1-93d7-add0fc9012e7 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:50.082256 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:50.106442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5840e2b-1a71-45fa-b565-36d27570bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:50.107124 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5840e2b-1a71-45fa-b565-36d27570bf14 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:50.107405 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5840e2b-1a71-45fa-b565-36d27570bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:50.107682 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5840e2b-1a71-45fa-b565-36d27570bf14 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:50.108400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5840e2b-1a71-45fa-b565-36d27570bf14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:50.109886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5840e2b-1a71-45fa-b565-36d27570bf14 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:50.132304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5f0ce-6d42-44ec-8ba0-61b28e1d77c6 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:50.142196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 238/467] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:26:49 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:50.142196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5840e2b-1a71-45fa-b565-36d27570bf14 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:50.144444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:50.187437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f78d6-2b12-4a9c-a293-c27acbd93304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:50.188577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f78d6-2b12-4a9c-a293-c27acbd93304 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:50.189166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f78d6-2b12-4a9c-a293-c27acbd93304 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:50.189459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f78d6-2b12-4a9c-a293-c27acbd93304 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:50.190295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f78d6-2b12-4a9c-a293-c27acbd93304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:50.190603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f78d6-2b12-4a9c-a293-c27acbd93304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:50.192296 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f78d6-2b12-4a9c-a293-c27acbd93304 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:50.214373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 230/468] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:50.215864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c1441-7109-4847-83e2-68556f527168 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:50.216155 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c1441-7109-4847-83e2-68556f527168 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:50.216432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c1441-7109-4847-83e2-68556f527168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:50.241361 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-521c1441-7109-4847-83e2-68556f527168 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:50.242547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-521c1441-7109-4847-83e2-68556f527168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:50.279653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5840e2b-1a71-45fa-b565-36d27570bf14 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:50.322826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e129a-1e3d-47b8-a6e2-cc882ea6d531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:50.324046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e129a-1e3d-47b8-a6e2-cc882ea6d531 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:50.324338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e129a-1e3d-47b8-a6e2-cc882ea6d531 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:50.324619 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e129a-1e3d-47b8-a6e2-cc882ea6d531 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:50.325545 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e129a-1e3d-47b8-a6e2-cc882ea6d531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:50.325647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e129a-1e3d-47b8-a6e2-cc882ea6d531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:50.327076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e129a-1e3d-47b8-a6e2-cc882ea6d531 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:50.333200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 231/469] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:50.338519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c1441-7109-4847-83e2-68556f527168 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:50.341386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 239/470] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:50.347567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e129a-1e3d-47b8-a6e2-cc882ea6d531 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:50.380684 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:50.466872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:50.470352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e129a-1e3d-47b8-a6e2-cc882ea6d531 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:50.474372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c1441-7109-4847-83e2-68556f527168 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:50.506709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882111e8-f7b5-4c9f-830c-113bac9c5320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:50.507662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882111e8-f7b5-4c9f-830c-113bac9c5320 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:50.508128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882111e8-f7b5-4c9f-830c-113bac9c5320 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:50.508587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882111e8-f7b5-4c9f-830c-113bac9c5320 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:50.509587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882111e8-f7b5-4c9f-830c-113bac9c5320 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:50.512262 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882111e8-f7b5-4c9f-830c-113bac9c5320 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:50.513093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11665109-0429-4ff6-888b-6246c2e369a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:50.513838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11665109-0429-4ff6-888b-6246c2e369a4 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:50.514218 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11665109-0429-4ff6-888b-6246c2e369a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:50.514566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11665109-0429-4ff6-888b-6246c2e369a4 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:50.515398 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11665109-0429-4ff6-888b-6246c2e369a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:50.517004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11665109-0429-4ff6-888b-6246c2e369a4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:50.539358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 240/471] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:26:50 2022] POST /identity/v3/users => generated 248 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:50.540620 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882111e8-f7b5-4c9f-830c-113bac9c5320 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:50.545649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 232/472] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:26:50 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:50.627265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1189f6-daed-468b-ba80-e3f936d4c85c None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:50.627659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1189f6-daed-468b-ba80-e3f936d4c85c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:50.628009 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1189f6-daed-468b-ba80-e3f936d4c85c None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:50.629820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 233/473] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:50.635776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c924e8-d268-4861-a01d-c9ce98050efe None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:50.636073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c924e8-d268-4861-a01d-c9ce98050efe None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:50.636479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c924e8-d268-4861-a01d-c9ce98050efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:50.648207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:50.659445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-d5c924e8-d268-4861-a01d-c9ce98050efe None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:50.661247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d5c924e8-d268-4861-a01d-c9ce98050efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:50.670320 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882111e8-f7b5-4c9f-830c-113bac9c5320 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:50.709858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9681c-53b2-475e-bef8-6b5080172b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:50.711177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9681c-53b2-475e-bef8-6b5080172b67 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:50.711637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9681c-53b2-475e-bef8-6b5080172b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:50.712092 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9681c-53b2-475e-bef8-6b5080172b67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:50.726121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9681c-53b2-475e-bef8-6b5080172b67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:50.727927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9681c-53b2-475e-bef8-6b5080172b67 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:50.744467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 234/474] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-cdf9681c-53b2-475e-bef8-6b5080172b67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:50.748935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:50.749801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:50.750696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:50.750696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:50.750696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:50.750696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:50.752803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 241/475] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:50.759042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c924e8-d268-4861-a01d-c9ce98050efe None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:50.806699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9681c-53b2-475e-bef8-6b5080172b67 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:50.872553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:50.894915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c924e8-d268-4861-a01d-c9ce98050efe None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:50.935813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9681c-53b2-475e-bef8-6b5080172b67 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:50.942527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7d1af-f2e0-40c2-a282-437e00df8014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:50.942788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7d1af-f2e0-40c2-a282-437e00df8014 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:50.943039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7d1af-f2e0-40c2-a282-437e00df8014 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:50.943373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7d1af-f2e0-40c2-a282-437e00df8014 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:50.944177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7d1af-f2e0-40c2-a282-437e00df8014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:50.944487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7d1af-f2e0-40c2-a282-437e00df8014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:50.946043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7d1af-f2e0-40c2-a282-437e00df8014 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:50.962511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 235/476] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:50.968882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7d1af-f2e0-40c2-a282-437e00df8014 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:50.976596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae752b2-7d6e-44bd-93c4-053df75d46ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:50.977574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae752b2-7d6e-44bd-93c4-053df75d46ef None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:50.978093 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae752b2-7d6e-44bd-93c4-053df75d46ef None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:50.978575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae752b2-7d6e-44bd-93c4-053df75d46ef None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:50.980123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae752b2-7d6e-44bd-93c4-053df75d46ef None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:50.982004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae752b2-7d6e-44bd-93c4-053df75d46ef None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:50.986890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:50.992167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.sql.core [None req-eae752b2-7d6e-44bd-93c4-053df75d46ef None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 12:26:50.992167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 12:26:50.992167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [parameters: {'id': '79dedd13aaa94cd3a771525ba5f02e62', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 12:26:50.992167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80327) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-eae752b2-7d6e-44bd-93c4-053df75d46ef None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application conn.query(q) Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result.read() Jul 25 12:26:51.013589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return ref.to_dict() Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 12:26:51.014517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application next(self.gen) Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application yield resource Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application next(self.gen) Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise self.value Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application session.commit() Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 12:26:51.015421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.session.flush() Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application compat.raise_( Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 12:26:51.016300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 12:26:51.017166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application conn.query(q) Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result.read() Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 12:26:51.018029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [parameters: {'id': '79dedd13aaa94cd3a771525ba5f02e62', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:51.018797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 12:26:51.019580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:51.028323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 242/477] 10.222.0.60 () {60 vars in 1131 bytes} [Mon Jul 25 12:26:50 2022] POST /identity/v3/roles => generated 144 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:26:51.031461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae752b2-7d6e-44bd-93c4-053df75d46ef None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:51.176740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae752b2-7d6e-44bd-93c4-053df75d46ef None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:51.228349 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34878cfb-bb00-4dba-823a-fdacaa29f623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:51.230290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34878cfb-bb00-4dba-823a-fdacaa29f623 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:51.231722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34878cfb-bb00-4dba-823a-fdacaa29f623 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:51.232235 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34878cfb-bb00-4dba-823a-fdacaa29f623 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:51.240012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7d1af-f2e0-40c2-a282-437e00df8014 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:51.254849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34878cfb-bb00-4dba-823a-fdacaa29f623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-34878cfb-bb00-4dba-823a-fdacaa29f623 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:51.262994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:51.263941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:51.265143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 243/478] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:26:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:51.274849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34878cfb-bb00-4dba-823a-fdacaa29f623 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:51.294517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d7ab1-14a4-4b85-afb3-f8dd18868cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:51.295525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d7ab1-14a4-4b85-afb3-f8dd18868cf4 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:51.296579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d7ab1-14a4-4b85-afb3-f8dd18868cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:51.297096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d7ab1-14a4-4b85-afb3-f8dd18868cf4 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:51.298069 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d7ab1-14a4-4b85-afb3-f8dd18868cf4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:51.299707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d7ab1-14a4-4b85-afb3-f8dd18868cf4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:51.312182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:51.332197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 236/479] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:26:50 2022] POST /identity/v3/users => generated 249 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:51.395402 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:51.442836 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34878cfb-bb00-4dba-823a-fdacaa29f623 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:51.482594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874426ef-2965-4067-b0e0-992dc062acb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:51.483638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874426ef-2965-4067-b0e0-992dc062acb0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:51.484150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874426ef-2965-4067-b0e0-992dc062acb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:51.484636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874426ef-2965-4067-b0e0-992dc062acb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:51.485646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874426ef-2965-4067-b0e0-992dc062acb0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:51.486392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874426ef-2965-4067-b0e0-992dc062acb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:51.488122 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874426ef-2965-4067-b0e0-992dc062acb0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:51.498262 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 244/480] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:52.536537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4b54c-f544-478a-a4db-12772466d40d None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:52.537035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4b54c-f544-478a-a4db-12772466d40d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:52.537035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4b54c-f544-478a-a4db-12772466d40d None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:52.538633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 237/481] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:52.544178 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04a982-53f5-463e-b47d-d160cae17fed None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:52.544981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04a982-53f5-463e-b47d-d160cae17fed None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:52.545470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04a982-53f5-463e-b47d-d160cae17fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:52.574892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0a04a982-53f5-463e-b47d-d160cae17fed None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:52.576412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0a04a982-53f5-463e-b47d-d160cae17fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:52.660495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 245/482] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:52.700306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4b54c-f544-478a-a4db-12772466d40d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:52.831535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4b54c-f544-478a-a4db-12772466d40d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:52.866067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994c44b-ac28-47a1-a225-626efdda7758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:52.866863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994c44b-ac28-47a1-a225-626efdda7758 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:52.867141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994c44b-ac28-47a1-a225-626efdda7758 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:52.867473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994c44b-ac28-47a1-a225-626efdda7758 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:52.878592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994c44b-ac28-47a1-a225-626efdda7758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-6994c44b-ac28-47a1-a225-626efdda7758 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:52.886295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:52.887193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:52.888284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 238/483] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:26:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:52.895228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04a982-53f5-463e-b47d-d160cae17fed None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:52.968815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:53.025633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04a982-53f5-463e-b47d-d160cae17fed None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:53.067615 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7721d43-270f-4378-bc8e-d94bd296e0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:53.068287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7721d43-270f-4378-bc8e-d94bd296e0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:53.068581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7721d43-270f-4378-bc8e-d94bd296e0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:53.068858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7721d43-270f-4378-bc8e-d94bd296e0c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:53.069613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7721d43-270f-4378-bc8e-d94bd296e0c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:53.069937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7721d43-270f-4378-bc8e-d94bd296e0c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:53.071327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7721d43-270f-4378-bc8e-d94bd296e0c9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:53.080127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 246/484] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:26:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:53.085484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994c44b-ac28-47a1-a225-626efdda7758 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:53.085809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85367875-346e-49cd-a44d-17622683c33c None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:53.086144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85367875-346e-49cd-a44d-17622683c33c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:53.086430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85367875-346e-49cd-a44d-17622683c33c None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:53.087960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 247/485] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:53.093709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901ca1f-fdca-4c16-983d-e3186696c1cc None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:53.094031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901ca1f-fdca-4c16-983d-e3186696c1cc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:53.094319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901ca1f-fdca-4c16-983d-e3186696c1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:53.129680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f901ca1f-fdca-4c16-983d-e3186696c1cc None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:53.130904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f901ca1f-fdca-4c16-983d-e3186696c1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:53.196129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:53.228745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 248/486] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:26:53.246491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994c44b-ac28-47a1-a225-626efdda7758 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:53.268976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901ca1f-fdca-4c16-983d-e3186696c1cc None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:53.288075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583c92f-22e4-41b1-a9a9-ad9da805333e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:53.288879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583c92f-22e4-41b1-a9a9-ad9da805333e None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:53.289165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583c92f-22e4-41b1-a9a9-ad9da805333e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:53.289445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583c92f-22e4-41b1-a9a9-ad9da805333e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:53.309763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583c92f-22e4-41b1-a9a9-ad9da805333e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:53.311282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583c92f-22e4-41b1-a9a9-ad9da805333e None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-1583c92f-22e4-41b1-a9a9-ad9da805333e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:53.333483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:53.334396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:53.335245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:53.335245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:53.335245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:53.335245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:53.335245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 239/487] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:53.336226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63ddd4-c366-4508-ac9d-0b158a7b7df4 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:53.336226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63ddd4-c366-4508-ac9d-0b158a7b7df4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:53.336498 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63ddd4-c366-4508-ac9d-0b158a7b7df4 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:53.342281 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 240/488] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:53.343864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63ddd4-c366-4508-ac9d-0b158a7b7df4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:53.348479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:53.408415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901ca1f-fdca-4c16-983d-e3186696c1cc None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:53.445732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9c4b0-710f-4c30-bae1-5352fb2f2978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:53.446908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9c4b0-710f-4c30-bae1-5352fb2f2978 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:53.447390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9c4b0-710f-4c30-bae1-5352fb2f2978 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:53.447953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9c4b0-710f-4c30-bae1-5352fb2f2978 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:53.458594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63ddd4-c366-4508-ac9d-0b158a7b7df4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:53.459600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9c4b0-710f-4c30-bae1-5352fb2f2978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:53.468045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-4ca9c4b0-710f-4c30-bae1-5352fb2f2978 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:53.468804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:53.469768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:53.471213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:53.471213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:53.471213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:53.471213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:53.471213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:53.471213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:53.471213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:53.471213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:53.471213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:53.471213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:53.471213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:26:53.471213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:53.473341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 249/489] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:26:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:53.474853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6c6fd-0acc-40b1-b52b-915585f63d41 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:53.475804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6c6fd-0acc-40b1-b52b-915585f63d41 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:53.476542 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6c6fd-0acc-40b1-b52b-915585f63d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:53.496563 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7ca6c6fd-0acc-40b1-b52b-915585f63d41 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:53.498188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7ca6c6fd-0acc-40b1-b52b-915585f63d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:53.499822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e820293-e517-40a1-a96d-60aadfc58d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:53.500796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e820293-e517-40a1-a96d-60aadfc58d63 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:53.501224 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e820293-e517-40a1-a96d-60aadfc58d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:53.501734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e820293-e517-40a1-a96d-60aadfc58d63 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:53.502682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e820293-e517-40a1-a96d-60aadfc58d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:53.503187 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e820293-e517-40a1-a96d-60aadfc58d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:53.504783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e820293-e517-40a1-a96d-60aadfc58d63 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:53.523073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 241/490] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:53.524278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e820293-e517-40a1-a96d-60aadfc58d63 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:53.573947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 250/491] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:53.573947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6c6fd-0acc-40b1-b52b-915585f63d41 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:53.607005 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:53.656979 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e820293-e517-40a1-a96d-60aadfc58d63 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:53.677121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6c6fd-0acc-40b1-b52b-915585f63d41 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:53.700773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348abe2-a98e-4947-8960-28f708ded015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:53.702381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348abe2-a98e-4947-8960-28f708ded015 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:53.702877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348abe2-a98e-4947-8960-28f708ded015 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:53.703553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348abe2-a98e-4947-8960-28f708ded015 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:53.704499 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348abe2-a98e-4947-8960-28f708ded015 None admin] RBAC: Adding query filter params (name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:53.704988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348abe2-a98e-4947-8960-28f708ded015 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:53.706659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348abe2-a98e-4947-8960-28f708ded015 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:53.715213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193eb8d4-fc87-4cfb-bedc-78112f6df51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:53.716093 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eb8d4-fc87-4cfb-bedc-78112f6df51d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:53.716448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eb8d4-fc87-4cfb-bedc-78112f6df51d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:53.716654 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 242/492] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:26:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:53.716812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eb8d4-fc87-4cfb-bedc-78112f6df51d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:53.717896 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348abe2-a98e-4947-8960-28f708ded015 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:53.725003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193eb8d4-fc87-4cfb-bedc-78112f6df51d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:53.726593 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193eb8d4-fc87-4cfb-bedc-78112f6df51d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-193eb8d4-fc87-4cfb-bedc-78112f6df51d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:26:53.743986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:53.745504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:53.746437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:53.746437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:53.746437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:26:53.746437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:26:53.746437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:26:53.746437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:26:53.746437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:26:53.746437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:26:53.746437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:26:53.746437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:53.750559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 251/493] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Jul 25 12:26:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:53.751569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193eb8d4-fc87-4cfb-bedc-78112f6df51d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:53.754663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:53.829583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348abe2-a98e-4947-8960-28f708ded015 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:53.860246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193eb8d4-fc87-4cfb-bedc-78112f6df51d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:53.862216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:53.866144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577357d6-e05e-4560-8775-99d04d85456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:53.867152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577357d6-e05e-4560-8775-99d04d85456d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:53.867616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577357d6-e05e-4560-8775-99d04d85456d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:53.868048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577357d6-e05e-4560-8775-99d04d85456d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:53.879786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577357d6-e05e-4560-8775-99d04d85456d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-577357d6-e05e-4560-8775-99d04d85456d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:53.888403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:53.889309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:53.890177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:53.890177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:53.890177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:53.890177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:53.890177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:53.890177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:53.890177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:53.890177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:53.890177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:53.890177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:53.890177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:26:53.890177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:53.897440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 243/494] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:26:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:53.898521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577357d6-e05e-4560-8775-99d04d85456d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:53.899851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9569d31-f535-46e8-92f4-9b70bc02bd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:53.900717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9569d31-f535-46e8-92f4-9b70bc02bd38 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:53.901098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9569d31-f535-46e8-92f4-9b70bc02bd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:53.901442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9569d31-f535-46e8-92f4-9b70bc02bd38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:53.912618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9569d31-f535-46e8-92f4-9b70bc02bd38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:53.914315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9569d31-f535-46e8-92f4-9b70bc02bd38 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c9569d31-f535-46e8-92f4-9b70bc02bd38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:53.935827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:53.936724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:53.937584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:53.937584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:53.937584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:53.937584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:53.937584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 252/495] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:53.937846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9569d31-f535-46e8-92f4-9b70bc02bd38 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:54.000192 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:54.011202 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577357d6-e05e-4560-8775-99d04d85456d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:54.048088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735d12d-19d6-4e6a-8595-9db428bbfa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:54.048763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735d12d-19d6-4e6a-8595-9db428bbfa2f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.048948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9569d31-f535-46e8-92f4-9b70bc02bd38 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:54.049053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735d12d-19d6-4e6a-8595-9db428bbfa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.049309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735d12d-19d6-4e6a-8595-9db428bbfa2f None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.050182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735d12d-19d6-4e6a-8595-9db428bbfa2f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:54.050498 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735d12d-19d6-4e6a-8595-9db428bbfa2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:54.051959 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735d12d-19d6-4e6a-8595-9db428bbfa2f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:54.086289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984df43-acb6-4e2e-aa51-35c83094575d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:54.087223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984df43-acb6-4e2e-aa51-35c83094575d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.087655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984df43-acb6-4e2e-aa51-35c83094575d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.088089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984df43-acb6-4e2e-aa51-35c83094575d None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.088873 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984df43-acb6-4e2e-aa51-35c83094575d None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:54.089190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984df43-acb6-4e2e-aa51-35c83094575d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:54.090616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984df43-acb6-4e2e-aa51-35c83094575d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:54.096502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 244/496] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Jul 25 12:26:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:54.096502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7c8ac-b5c4-44f7-af2a-a0a97cfe2ce0 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.096502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7c8ac-b5c4-44f7-af2a-a0a97cfe2ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.096502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7c8ac-b5c4-44f7-af2a-a0a97cfe2ce0 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.096502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 245/497] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:54.096502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7c8ac-b5c4-44f7-af2a-a0a97cfe2ce0 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:54.112731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 253/498] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:26:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:54.112731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984df43-acb6-4e2e-aa51-35c83094575d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:54.124576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:54.201319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7c8ac-b5c4-44f7-af2a-a0a97cfe2ce0 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:54.225544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984df43-acb6-4e2e-aa51-35c83094575d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:54.234665 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e61088-a8c4-455a-afa0-9b04c3816269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:54.235324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e61088-a8c4-455a-afa0-9b04c3816269 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.235603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e61088-a8c4-455a-afa0-9b04c3816269 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.235875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e61088-a8c4-455a-afa0-9b04c3816269 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.236591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e61088-a8c4-455a-afa0-9b04c3816269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:54.236890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e61088-a8c4-455a-afa0-9b04c3816269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:54.238288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e61088-a8c4-455a-afa0-9b04c3816269 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:54.249687 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 246/499] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:54.250895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e0c52-90d4-4d47-91c9-ef8839e43993 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.251185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e0c52-90d4-4d47-91c9-ef8839e43993 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.251464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e0c52-90d4-4d47-91c9-ef8839e43993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.259894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba405ea-d363-4af3-9b20-0ba67379cdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:54.260577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba405ea-d363-4af3-9b20-0ba67379cdac None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.260859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba405ea-d363-4af3-9b20-0ba67379cdac None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.261133 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba405ea-d363-4af3-9b20-0ba67379cdac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.268663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-524e0c52-90d4-4d47-91c9-ef8839e43993 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:54.269849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-524e0c52-90d4-4d47-91c9-ef8839e43993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:54.271890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:54.327664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba405ea-d363-4af3-9b20-0ba67379cdac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:54.329108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba405ea-d363-4af3-9b20-0ba67379cdac None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:54.335964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 247/500] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:54.336787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e0c52-90d4-4d47-91c9-ef8839e43993 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:54.352625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:54.433278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e0c52-90d4-4d47-91c9-ef8839e43993 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:54.442633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 254/501] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:54.444088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f1608-0201-487c-ad3e-d15b120acad7 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.444374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f1608-0201-487c-ad3e-d15b120acad7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.444651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f1608-0201-487c-ad3e-d15b120acad7 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.445943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 255/502] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:54.446625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f1608-0201-487c-ad3e-d15b120acad7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:54.470813 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92811af3-1a71-4e61-b40b-3c7d283b3def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:54.471617 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92811af3-1a71-4e61-b40b-3c7d283b3def None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.471876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92811af3-1a71-4e61-b40b-3c7d283b3def None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.472149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92811af3-1a71-4e61-b40b-3c7d283b3def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.478573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:54.483930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92811af3-1a71-4e61-b40b-3c7d283b3def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:54.485394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92811af3-1a71-4e61-b40b-3c7d283b3def None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-92811af3-1a71-4e61-b40b-3c7d283b3def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:54.502566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:54.503452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:54.504329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:54.504329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:54.504329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:54.504329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:54.504329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 248/503] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:54.504600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92811af3-1a71-4e61-b40b-3c7d283b3def None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:54.552486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f1608-0201-487c-ad3e-d15b120acad7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:54.561387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:54.585900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fefdb-dbb8-4844-b607-189dcf395d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:54.586705 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fefdb-dbb8-4844-b607-189dcf395d57 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.586985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fefdb-dbb8-4844-b607-189dcf395d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.587654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fefdb-dbb8-4844-b607-189dcf395d57 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.595142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483fefdb-dbb8-4844-b607-189dcf395d57 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:54.596613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483fefdb-dbb8-4844-b607-189dcf395d57 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:54.599139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92811af3-1a71-4e61-b40b-3c7d283b3def None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-483fefdb-dbb8-4844-b607-189dcf395d57 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:26:54.611815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:54.612848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:54.614143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:54.614143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:54.614143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:26:54.614143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:26:54.614143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:26:54.614143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:26:54.614143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:26:54.614143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:26:54.614143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:26:54.614143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:54.614143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 256/504] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:54.614143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fefdb-dbb8-4844-b607-189dcf395d57 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:54.636109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de2c34-9dc8-4fa0-9455-bca31e259502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:54.636895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de2c34-9dc8-4fa0-9455-bca31e259502 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.637174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de2c34-9dc8-4fa0-9455-bca31e259502 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.637445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de2c34-9dc8-4fa0-9455-bca31e259502 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.647168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de2c34-9dc8-4fa0-9455-bca31e259502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-32de2c34-9dc8-4fa0-9455-bca31e259502 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:54.655383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:54.656248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:54.657020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:54.657020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:54.657020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:54.657020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:54.657020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:54.657020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:54.657020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:54.657020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:54.657020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:54.657020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:54.657020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:26:54.657020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:54.658535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 249/505] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:26:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:54.659744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9341e5-7157-4ba2-bc1a-7926228afc2c None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.660032 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9341e5-7157-4ba2-bc1a-7926228afc2c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.660673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9341e5-7157-4ba2-bc1a-7926228afc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.677752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-8f9341e5-7157-4ba2-bc1a-7926228afc2c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:54.679019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8f9341e5-7157-4ba2-bc1a-7926228afc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:54.712127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fefdb-dbb8-4844-b607-189dcf395d57 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:54.726311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:54.745939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64262a43-854d-412e-b96b-0f4202a70abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:54.746727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64262a43-854d-412e-b96b-0f4202a70abf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.746843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 250/506] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:54.747040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64262a43-854d-412e-b96b-0f4202a70abf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.747275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64262a43-854d-412e-b96b-0f4202a70abf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.747579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9341e5-7157-4ba2-bc1a-7926228afc2c None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:54.757292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64262a43-854d-412e-b96b-0f4202a70abf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:54.758806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64262a43-854d-412e-b96b-0f4202a70abf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-64262a43-854d-412e-b96b-0f4202a70abf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:54.778800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:54.779708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:54.780556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:54.780556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:54.780556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:54.780556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:54.782168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 257/507] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:54.783094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64262a43-854d-412e-b96b-0f4202a70abf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:54.804881 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:54.854842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9341e5-7157-4ba2-bc1a-7926228afc2c None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:54.883082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64262a43-854d-412e-b96b-0f4202a70abf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:54.887551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9fe15-1de1-4205-879c-106121a9ec52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:54.888206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9fe15-1de1-4205-879c-106121a9ec52 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.888486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9fe15-1de1-4205-879c-106121a9ec52 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.888760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9fe15-1de1-4205-879c-106121a9ec52 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.889477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9fe15-1de1-4205-879c-106121a9ec52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:54.889782 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9fe15-1de1-4205-879c-106121a9ec52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:54.891191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9fe15-1de1-4205-879c-106121a9ec52 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:54.902723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 251/508] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:54.903506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9fe15-1de1-4205-879c-106121a9ec52 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:54.916748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433b8ed-9904-43c4-8daa-46337aabaa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:54.917412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433b8ed-9904-43c4-8daa-46337aabaa86 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:54.917702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433b8ed-9904-43c4-8daa-46337aabaa86 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:54.918006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433b8ed-9904-43c4-8daa-46337aabaa86 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:54.918853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433b8ed-9904-43c4-8daa-46337aabaa86 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:54.919223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433b8ed-9904-43c4-8daa-46337aabaa86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:54.920867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433b8ed-9904-43c4-8daa-46337aabaa86 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:54.945805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:54.959837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 258/509] 10.222.0.60 () {56 vars in 1137 bytes} [Mon Jul 25 12:26:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:54.960677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433b8ed-9904-43c4-8daa-46337aabaa86 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:55.011929 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9fe15-1de1-4205-879c-106121a9ec52 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:55.038503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:55.044295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f91fe-e9f2-462d-9975-992e511b3177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:55.044956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f91fe-e9f2-462d-9975-992e511b3177 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:55.045235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f91fe-e9f2-462d-9975-992e511b3177 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:55.045512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f91fe-e9f2-462d-9975-992e511b3177 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:55.046277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f91fe-e9f2-462d-9975-992e511b3177 None admin] RBAC: Adding query filter params (name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:55.046582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f91fe-e9f2-462d-9975-992e511b3177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:55.047934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f91fe-e9f2-462d-9975-992e511b3177 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:55.055619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 252/510] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:26:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:55.056383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f91fe-e9f2-462d-9975-992e511b3177 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:55.060755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433b8ed-9904-43c4-8daa-46337aabaa86 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:55.095597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e196d-293c-4c49-a487-0ab1f96cb826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:55.096387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e196d-293c-4c49-a487-0ab1f96cb826 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:55.096674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e196d-293c-4c49-a487-0ab1f96cb826 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:55.096955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e196d-293c-4c49-a487-0ab1f96cb826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:55.107779 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e196d-293c-4c49-a487-0ab1f96cb826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:55.109233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e196d-293c-4c49-a487-0ab1f96cb826 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-271e196d-293c-4c49-a487-0ab1f96cb826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:55.126420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:55.127321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:55.128151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:55.128151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:55.128151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:55.128151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:55.128151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 259/511] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:55.128405 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e196d-293c-4c49-a487-0ab1f96cb826 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:55.156250 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f91fe-e9f2-462d-9975-992e511b3177 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:55.157742 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:55.192199 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8093db-043b-406c-8292-2b9978ac5170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:55.192861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8093db-043b-406c-8292-2b9978ac5170 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:55.193141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8093db-043b-406c-8292-2b9978ac5170 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:55.193415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8093db-043b-406c-8292-2b9978ac5170 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:55.194161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8093db-043b-406c-8292-2b9978ac5170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:55.194470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8093db-043b-406c-8292-2b9978ac5170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:55.195837 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8093db-043b-406c-8292-2b9978ac5170 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:55.207238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 253/512] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:55.208024 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8093db-043b-406c-8292-2b9978ac5170 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:55.228701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e196d-293c-4c49-a487-0ab1f96cb826 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:55.258897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:55.265486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9fbb8-ebf8-4684-9a6b-9d2f7f68a29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:55.266238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9fbb8-ebf8-4684-9a6b-9d2f7f68a29f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:55.266525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9fbb8-ebf8-4684-9a6b-9d2f7f68a29f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:55.266797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9fbb8-ebf8-4684-9a6b-9d2f7f68a29f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:55.273811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9fbb8-ebf8-4684-9a6b-9d2f7f68a29f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:55.275284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9fbb8-ebf8-4684-9a6b-9d2f7f68a29f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-83a9fbb8-ebf8-4684-9a6b-9d2f7f68a29f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:26:55.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:55.290653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:55.291535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:55.291535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:55.291535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:26:55.291535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:26:55.291535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:26:55.291535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:26:55.291535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:26:55.291535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:26:55.291535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 12:26:55.291535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:55.291535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 260/513] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:26:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:55.292642 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9fbb8-ebf8-4684-9a6b-9d2f7f68a29f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:55.314230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8093db-043b-406c-8292-2b9978ac5170 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:55.347469 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434974b-af05-4fa5-b7f7-49122fdd1607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:55.348227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434974b-af05-4fa5-b7f7-49122fdd1607 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:55.348513 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434974b-af05-4fa5-b7f7-49122fdd1607 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:55.348787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434974b-af05-4fa5-b7f7-49122fdd1607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:55.377881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:55.391168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9fbb8-ebf8-4684-9a6b-9d2f7f68a29f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:55.415746 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434974b-af05-4fa5-b7f7-49122fdd1607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:55.417205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434974b-af05-4fa5-b7f7-49122fdd1607 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:55.424629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03017fe7-3256-486e-a1dc-2c1715ef2905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:55.425397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03017fe7-3256-486e-a1dc-2c1715ef2905 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:55.425675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03017fe7-3256-486e-a1dc-2c1715ef2905 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:55.425968 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03017fe7-3256-486e-a1dc-2c1715ef2905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:55.435578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03017fe7-3256-486e-a1dc-2c1715ef2905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:55.436990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03017fe7-3256-486e-a1dc-2c1715ef2905 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-03017fe7-3256-486e-a1dc-2c1715ef2905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:55.454650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:55.455491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:55.456336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:55.456336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:55.456336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:55.456336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:55.456336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 261/514] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:55.456602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03017fe7-3256-486e-a1dc-2c1715ef2905 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:55.498242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:55.516778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 254/515] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:55.517551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434974b-af05-4fa5-b7f7-49122fdd1607 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:55.554517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03017fe7-3256-486e-a1dc-2c1715ef2905 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:55.575553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:55.587463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e4243-f50e-4950-b1cd-6427786adccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:55.588121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e4243-f50e-4950-b1cd-6427786adccc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:55.588405 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e4243-f50e-4950-b1cd-6427786adccc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:55.589247 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e4243-f50e-4950-b1cd-6427786adccc None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:55.589999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e4243-f50e-4950-b1cd-6427786adccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:55.590309 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e4243-f50e-4950-b1cd-6427786adccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:55.591683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e4243-f50e-4950-b1cd-6427786adccc None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:55.603655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 262/516] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:55.604451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e4243-f50e-4950-b1cd-6427786adccc None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:55.614879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434974b-af05-4fa5-b7f7-49122fdd1607 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:55.648393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148141ab-dacb-48dd-a2ed-6e40c867288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:55.649176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148141ab-dacb-48dd-a2ed-6e40c867288b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:55.649456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148141ab-dacb-48dd-a2ed-6e40c867288b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:55.649733 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148141ab-dacb-48dd-a2ed-6e40c867288b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:55.656969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148141ab-dacb-48dd-a2ed-6e40c867288b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:55.658454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148141ab-dacb-48dd-a2ed-6e40c867288b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-148141ab-dacb-48dd-a2ed-6e40c867288b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:26:55.672891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:55.673893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:55.674807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:26:55.674807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:26:55.674807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:26:55.674807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:26:55.674807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:26:55.674807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:26:55.674807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:26:55.674807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:55.674807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 255/517] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:55.675297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148141ab-dacb-48dd-a2ed-6e40c867288b None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:55.703398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e4243-f50e-4950-b1cd-6427786adccc None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:55.704986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:55.739243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaed6f-988a-4299-84b5-b66c1f8f02ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:55.739904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaed6f-988a-4299-84b5-b66c1f8f02ef None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:55.740180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaed6f-988a-4299-84b5-b66c1f8f02ef None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:55.740452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaed6f-988a-4299-84b5-b66c1f8f02ef None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:55.741280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeaed6f-988a-4299-84b5-b66c1f8f02ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:55.741580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeaed6f-988a-4299-84b5-b66c1f8f02ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:55.743071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeaed6f-988a-4299-84b5-b66c1f8f02ef None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:55.760205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 263/518] 10.222.0.60 () {56 vars in 1137 bytes} [Mon Jul 25 12:26:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:55.761729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaed6f-988a-4299-84b5-b66c1f8f02ef None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:55.777391 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148141ab-dacb-48dd-a2ed-6e40c867288b None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:55.798791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:55.811728 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc5844-928d-4614-982f-f89ccd893241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:55.812381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc5844-928d-4614-982f-f89ccd893241 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:55.812663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc5844-928d-4614-982f-f89ccd893241 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:55.812941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc5844-928d-4614-982f-f89ccd893241 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:55.813647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc5844-928d-4614-982f-f89ccd893241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:55.813985 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc5844-928d-4614-982f-f89ccd893241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:55.815356 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc5844-928d-4614-982f-f89ccd893241 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:55.827327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 256/519] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:55.828084 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc5844-928d-4614-982f-f89ccd893241 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:55.862843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaed6f-988a-4299-84b5-b66c1f8f02ef None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:55.895754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf541d7d-b2f6-44fa-b3fa-ac9c05d84431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:55.896541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf541d7d-b2f6-44fa-b3fa-ac9c05d84431 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:55.896817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf541d7d-b2f6-44fa-b3fa-ac9c05d84431 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:55.897088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf541d7d-b2f6-44fa-b3fa-ac9c05d84431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:55.907438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf541d7d-b2f6-44fa-b3fa-ac9c05d84431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:55.908879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf541d7d-b2f6-44fa-b3fa-ac9c05d84431 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:55.923737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc5844-928d-4614-982f-f89ccd893241 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-cf541d7d-b2f6-44fa-b3fa-ac9c05d84431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:55.926321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:55.927171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:55.928039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:55.928039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:55.928039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:55.928039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:55.928039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 264/520] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:55.928717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf541d7d-b2f6-44fa-b3fa-ac9c05d84431 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:55.944389 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:55.962190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcf93d-b7b7-454a-b110-243976c1ecdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:55.962850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcf93d-b7b7-454a-b110-243976c1ecdc None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:55.963127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcf93d-b7b7-454a-b110-243976c1ecdc None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:55.963398 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcf93d-b7b7-454a-b110-243976c1ecdc None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:55.964116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcf93d-b7b7-454a-b110-243976c1ecdc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:55.965528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcf93d-b7b7-454a-b110-243976c1ecdc None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:55.986114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 257/521] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:26:55 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:26:55.986916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcf93d-b7b7-454a-b110-243976c1ecdc None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:56.033820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:56.058066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf541d7d-b2f6-44fa-b3fa-ac9c05d84431 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:56.100930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721506d-9f17-4715-864e-5b58ddc80b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:56.101704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721506d-9f17-4715-864e-5b58ddc80b38 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:56.102071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721506d-9f17-4715-864e-5b58ddc80b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:56.102453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721506d-9f17-4715-864e-5b58ddc80b38 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:56.103313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721506d-9f17-4715-864e-5b58ddc80b38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:56.103628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721506d-9f17-4715-864e-5b58ddc80b38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:56.104141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcf93d-b7b7-454a-b110-243976c1ecdc None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:56.105098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721506d-9f17-4715-864e-5b58ddc80b38 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:56.126115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 265/522] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:26:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:56.127170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721506d-9f17-4715-864e-5b58ddc80b38 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:56.145326 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2464cc5-4a88-48dd-9626-b8eaad6d613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:56.146020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2464cc5-4a88-48dd-9626-b8eaad6d613c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:56.146308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2464cc5-4a88-48dd-9626-b8eaad6d613c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:56.147182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2464cc5-4a88-48dd-9626-b8eaad6d613c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:56.220632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:56.227058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2464cc5-4a88-48dd-9626-b8eaad6d613c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:56.228514 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2464cc5-4a88-48dd-9626-b8eaad6d613c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:56.255390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721506d-9f17-4715-864e-5b58ddc80b38 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:56.270276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:56.295382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6412b9d-a6e6-44b9-9fb2-581ec8bcf034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:56.296417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6412b9d-a6e6-44b9-9fb2-581ec8bcf034 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:56.296843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6412b9d-a6e6-44b9-9fb2-581ec8bcf034 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:56.297263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6412b9d-a6e6-44b9-9fb2-581ec8bcf034 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:56.305548 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6412b9d-a6e6-44b9-9fb2-581ec8bcf034 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:56.307287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6412b9d-a6e6-44b9-9fb2-581ec8bcf034 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b6412b9d-a6e6-44b9-9fb2-581ec8bcf034 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:26:56.322377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:56.323329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:56.324352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:56.324352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:56.324352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:26:56.324352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:26:56.324352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:26:56.324352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:26:56.324352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:26:56.324352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:26:56.324352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 12:26:56.324352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:56.324352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 266/523] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:26:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:56.324352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6412b9d-a6e6-44b9-9fb2-581ec8bcf034 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:56.344249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 258/524] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:56.349291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2464cc5-4a88-48dd-9626-b8eaad6d613c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:56.430889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6412b9d-a6e6-44b9-9fb2-581ec8bcf034 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:56.453936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2464cc5-4a88-48dd-9626-b8eaad6d613c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:56.460586 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:56.468714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7f85e-6649-4118-84f8-5bf3caa597e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:56.469384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7f85e-6649-4118-84f8-5bf3caa597e9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:56.469664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7f85e-6649-4118-84f8-5bf3caa597e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:56.469961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7f85e-6649-4118-84f8-5bf3caa597e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:56.470678 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7f85e-6649-4118-84f8-5bf3caa597e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:56.470984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7f85e-6649-4118-84f8-5bf3caa597e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:56.472424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7f85e-6649-4118-84f8-5bf3caa597e9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:56.485389 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 267/525] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:56.486406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7f85e-6649-4118-84f8-5bf3caa597e9 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:56.497143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97e03d-b527-4759-aa5f-b63b8fc15390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:56.497805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97e03d-b527-4759-aa5f-b63b8fc15390 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:56.498120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97e03d-b527-4759-aa5f-b63b8fc15390 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:56.498397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97e03d-b527-4759-aa5f-b63b8fc15390 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:56.499236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97e03d-b527-4759-aa5f-b63b8fc15390 None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=ccf2dedaf7a449d2b51b9e9f3b420091) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:56.499552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97e03d-b527-4759-aa5f-b63b8fc15390 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:56.501029 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97e03d-b527-4759-aa5f-b63b8fc15390 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:56.513951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 259/526] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:56 2022] GET /identity/v3/role_assignments?user.id=ccf2dedaf7a449d2b51b9e9f3b420091&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:56.513951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97e03d-b527-4759-aa5f-b63b8fc15390 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:56.513951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:56.637177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7f85e-6649-4118-84f8-5bf3caa597e9 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:56.653637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97e03d-b527-4759-aa5f-b63b8fc15390 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:56.684030 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827a181-7ec1-4807-ac57-740ff74f3be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:56.685160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827a181-7ec1-4807-ac57-740ff74f3be3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:56.685520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827a181-7ec1-4807-ac57-740ff74f3be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:56.685867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827a181-7ec1-4807-ac57-740ff74f3be3 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:56.686744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827a181-7ec1-4807-ac57-740ff74f3be3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:56.687056 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827a181-7ec1-4807-ac57-740ff74f3be3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:56.688521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827a181-7ec1-4807-ac57-740ff74f3be3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:56.694017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e9109-485c-4104-890b-c328b91c5a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:56.694941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e9109-485c-4104-890b-c328b91c5a4f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:56.695313 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e9109-485c-4104-890b-c328b91c5a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:56.695682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e9109-485c-4104-890b-c328b91c5a4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:56.707327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e9109-485c-4104-890b-c328b91c5a4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:56.708011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 268/527] 10.222.0.60 () {56 vars in 1139 bytes} [Mon Jul 25 12:26:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:56.708988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827a181-7ec1-4807-ac57-740ff74f3be3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:56.714174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e9109-485c-4104-890b-c328b91c5a4f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-cb4e9109-485c-4104-890b-c328b91c5a4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:56.733013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:56.733956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:56.734815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:56.734815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:56.734815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:56.734815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:56.738842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 260/528] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:56.739771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e9109-485c-4104-890b-c328b91c5a4f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:56.774261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:56.810367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:56.841981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827a181-7ec1-4807-ac57-740ff74f3be3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:56.857056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e9109-485c-4104-890b-c328b91c5a4f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:56.878495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73915f71-c2db-400a-9c73-fcdf62797a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:56.879287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73915f71-c2db-400a-9c73-fcdf62797a91 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:56.879570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73915f71-c2db-400a-9c73-fcdf62797a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:56.879842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73915f71-c2db-400a-9c73-fcdf62797a91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:56.888115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73915f71-c2db-400a-9c73-fcdf62797a91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:56.889602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73915f71-c2db-400a-9c73-fcdf62797a91 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:56.899698 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362f98b-c0fb-4bad-8c03-997872afb137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:56.900623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362f98b-c0fb-4bad-8c03-997872afb137 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:56.901183 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362f98b-c0fb-4bad-8c03-997872afb137 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:56.901677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362f98b-c0fb-4bad-8c03-997872afb137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-73915f71-c2db-400a-9c73-fcdf62797a91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:26:56.904568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:56.905418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:56.906303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:26:56.906303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:26:56.906303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:26:56.906303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:26:56.906303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:26:56.906303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:26:56.906303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:26:56.906303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:56.910480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 269/529] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:56.911400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73915f71-c2db-400a-9c73-fcdf62797a91 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:56.980299 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362f98b-c0fb-4bad-8c03-997872afb137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:56.982451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362f98b-c0fb-4bad-8c03-997872afb137 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:57.021480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73915f71-c2db-400a-9c73-fcdf62797a91 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:57.032486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:57.042921 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:57.061078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1c205-4939-4462-b308-8fde3a18309c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:57.061266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1c205-4939-4462-b308-8fde3a18309c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:57.061512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1c205-4939-4462-b308-8fde3a18309c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:57.061739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1c205-4939-4462-b308-8fde3a18309c None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:57.062498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1c205-4939-4462-b308-8fde3a18309c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:57.062802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1c205-4939-4462-b308-8fde3a18309c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:57.064322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1c205-4939-4462-b308-8fde3a18309c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:57.081769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 270/530] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:57.082699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1c205-4939-4462-b308-8fde3a18309c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:57.097329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 261/531] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:26:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:26:57.098561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362f98b-c0fb-4bad-8c03-997872afb137 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:57.198180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1c205-4939-4462-b308-8fde3a18309c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:57.220671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362f98b-c0fb-4bad-8c03-997872afb137 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:57.235518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869678d-02b0-49c8-b701-e7ebd4c55740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:57.236083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869678d-02b0-49c8-b701-e7ebd4c55740 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:57.236362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869678d-02b0-49c8-b701-e7ebd4c55740 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:57.236633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869678d-02b0-49c8-b701-e7ebd4c55740 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:57.237420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869678d-02b0-49c8-b701-e7ebd4c55740 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:57.237726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869678d-02b0-49c8-b701-e7ebd4c55740 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:57.239378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869678d-02b0-49c8-b701-e7ebd4c55740 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:57.253016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 271/532] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:26:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:57.258389 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869678d-02b0-49c8-b701-e7ebd4c55740 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:57.262367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df28b62-58ea-40d3-b590-3654ccd237aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:57.263426 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df28b62-58ea-40d3-b590-3654ccd237aa None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:57.263905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df28b62-58ea-40d3-b590-3654ccd237aa None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:57.264395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df28b62-58ea-40d3-b590-3654ccd237aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:57.272165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df28b62-58ea-40d3-b590-3654ccd237aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:57.273991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df28b62-58ea-40d3-b590-3654ccd237aa None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:57.289390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-8df28b62-58ea-40d3-b590-3654ccd237aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:26:57.290793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:57.291676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:57.293046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:26:57.293046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:26:57.293046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:26:57.293046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:26:57.293046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:26:57.293046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:26:57.293046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:26:57.293046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:57.296287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 262/533] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:57.297362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df28b62-58ea-40d3-b590-3654ccd237aa None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:57.318306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:57.365945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869678d-02b0-49c8-b701-e7ebd4c55740 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:57.404149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead024e-e34b-4815-9b67-70dc7b12e5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:57.404945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead024e-e34b-4815-9b67-70dc7b12e5eb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:57.405225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead024e-e34b-4815-9b67-70dc7b12e5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:57.405499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead024e-e34b-4815-9b67-70dc7b12e5eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:57.406328 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df28b62-58ea-40d3-b590-3654ccd237aa None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:57.420342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead024e-e34b-4815-9b67-70dc7b12e5eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:57.421799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead024e-e34b-4815-9b67-70dc7b12e5eb None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:57.445403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e267f80-407e-485f-8460-cd900062d135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:57.446015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e267f80-407e-485f-8460-cd900062d135 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:57.446297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e267f80-407e-485f-8460-cd900062d135 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:57.446571 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e267f80-407e-485f-8460-cd900062d135 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-9ead024e-e34b-4815-9b67-70dc7b12e5eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:57.447312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:57.448152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:57.449013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:57.449013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:57.449013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:57.449013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:57.449230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e267f80-407e-485f-8460-cd900062d135 None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=eb4c5027951c4c55bd826943cec15309) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:57.449230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e267f80-407e-485f-8460-cd900062d135 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:57.449230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e267f80-407e-485f-8460-cd900062d135 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:57.451361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 272/534] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:57.453154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead024e-e34b-4815-9b67-70dc7b12e5eb None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:57.461093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 263/535] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:57 2022] GET /identity/v3/role_assignments?user.id=eb4c5027951c4c55bd826943cec15309&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:57.461093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e267f80-407e-485f-8460-cd900062d135 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:57.612863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:57.617503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:57.655457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead024e-e34b-4815-9b67-70dc7b12e5eb None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:57.658244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e267f80-407e-485f-8460-cd900062d135 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:57.706524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e0812-be97-49f0-8db9-954d36589674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:57.707116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e0812-be97-49f0-8db9-954d36589674 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:57.707386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e0812-be97-49f0-8db9-954d36589674 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:57.707665 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e0812-be97-49f0-8db9-954d36589674 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:57.708504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e0812-be97-49f0-8db9-954d36589674 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:57.708768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e0812-be97-49f0-8db9-954d36589674 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:57.710473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e0812-be97-49f0-8db9-954d36589674 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:57.714418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a9ae2-51db-4a1a-831e-27c681a719c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:57.715635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a9ae2-51db-4a1a-831e-27c681a719c7 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:57.716144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a9ae2-51db-4a1a-831e-27c681a719c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:57.716611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a9ae2-51db-4a1a-831e-27c681a719c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:57.717525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a9ae2-51db-4a1a-831e-27c681a719c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:57.718053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a9ae2-51db-4a1a-831e-27c681a719c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:57.720211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a9ae2-51db-4a1a-831e-27c681a719c7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:57.724976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 273/536] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:26:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:57.735323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e0812-be97-49f0-8db9-954d36589674 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:57.735574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 264/537] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:57.740621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a9ae2-51db-4a1a-831e-27c681a719c7 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:57.871092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a9ae2-51db-4a1a-831e-27c681a719c7 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:57.910926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db5a72-fd7b-4cce-8a63-b25bc94b2bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:57.911983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db5a72-fd7b-4cce-8a63-b25bc94b2bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:57.912404 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db5a72-fd7b-4cce-8a63-b25bc94b2bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:57.912815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db5a72-fd7b-4cce-8a63-b25bc94b2bf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:57.920791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db5a72-fd7b-4cce-8a63-b25bc94b2bf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:57.922619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db5a72-fd7b-4cce-8a63-b25bc94b2bf3 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:57.924495 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-31db5a72-fd7b-4cce-8a63-b25bc94b2bf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:26:57.939532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:57.940561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:57.941614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:26:57.941614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:26:57.941614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:26:57.941614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:26:57.941614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:26:57.941614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:26:57.941614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:26:57.941614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:57.944204 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 265/538] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:26:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:57.954836 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db5a72-fd7b-4cce-8a63-b25bc94b2bf3 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:57.991112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e0812-be97-49f0-8db9-954d36589674 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:58.079071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187f232-b6e3-4d99-bab6-bb6404853397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:58.080069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187f232-b6e3-4d99-bab6-bb6404853397 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:58.081592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187f232-b6e3-4d99-bab6-bb6404853397 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:58.082598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187f232-b6e3-4d99-bab6-bb6404853397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:58.083761 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187f232-b6e3-4d99-bab6-bb6404853397 None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=6b07dcd8c0de4d1694d4363520726df2) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:58.084820 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187f232-b6e3-4d99-bab6-bb6404853397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:58.087744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187f232-b6e3-4d99-bab6-bb6404853397 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:58.100008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db5a72-fd7b-4cce-8a63-b25bc94b2bf3 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:58.101566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 274/539] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:57 2022] GET /identity/v3/role_assignments?user.id=6b07dcd8c0de4d1694d4363520726df2&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 259 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:58.171796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874488ad-aad7-4071-ae55-d26dce2b942c None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:58.171796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874488ad-aad7-4071-ae55-d26dce2b942c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:58.171796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874488ad-aad7-4071-ae55-d26dce2b942c None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:58.173875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 275/540] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:58.174794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98e68-a92c-4982-bf16-5fe9e7684231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:58.175576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98e68-a92c-4982-bf16-5fe9e7684231 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:58.175953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98e68-a92c-4982-bf16-5fe9e7684231 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:58.176322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98e68-a92c-4982-bf16-5fe9e7684231 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:58.177240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b98e68-a92c-4982-bf16-5fe9e7684231 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:58.177680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b98e68-a92c-4982-bf16-5fe9e7684231 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:58.179310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b98e68-a92c-4982-bf16-5fe9e7684231 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:58.183525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a90160-f413-44ce-8c44-cac696454eb8 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:58.183941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a90160-f413-44ce-8c44-cac696454eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:58.184314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a90160-f413-44ce-8c44-cac696454eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:58.209548 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:58.225206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b4a90160-f413-44ce-8c44-cac696454eb8 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:58.227872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b4a90160-f413-44ce-8c44-cac696454eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:58.234572 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 266/541] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:26:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:58.240671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98e68-a92c-4982-bf16-5fe9e7684231 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:58.331089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 276/542] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:58.346741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:58.371403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a90160-f413-44ce-8c44-cac696454eb8 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:58.392295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98e68-a92c-4982-bf16-5fe9e7684231 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:58.428749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957553f-e908-4d41-9b51-cf6e493461d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:58.429554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957553f-e908-4d41-9b51-cf6e493461d7 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:58.429957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957553f-e908-4d41-9b51-cf6e493461d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:58.430341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957553f-e908-4d41-9b51-cf6e493461d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:58.431294 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957553f-e908-4d41-9b51-cf6e493461d7 None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=375d321d6de2494794254e50151c02c4) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:58.431706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957553f-e908-4d41-9b51-cf6e493461d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:58.433381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957553f-e908-4d41-9b51-cf6e493461d7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:58.439559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 267/543] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:26:58 2022] GET /identity/v3/role_assignments?user.id=375d321d6de2494794254e50151c02c4&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:58.556530 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a90160-f413-44ce-8c44-cac696454eb8 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:58.608796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f6f07-e046-486e-bc7d-163cf026dc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:58.609782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f6f07-e046-486e-bc7d-163cf026dc16 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:58.610132 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f6f07-e046-486e-bc7d-163cf026dc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:58.610421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f6f07-e046-486e-bc7d-163cf026dc16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:58.616732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:58.631195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f6f07-e046-486e-bc7d-163cf026dc16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-319f6f07-e046-486e-bc7d-163cf026dc16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:58.642791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:58.643743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:58.645299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 277/544] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:26:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:26:58.653506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957553f-e908-4d41-9b51-cf6e493461d7 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:58.667008 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4650c9-ccc5-414a-8f39-21081b3fc337 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:58.667630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4650c9-ccc5-414a-8f39-21081b3fc337 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:58.667630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4650c9-ccc5-414a-8f39-21081b3fc337 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:58.671778 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 278/545] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:58.676381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e120f4-f04e-4efb-a0b7-cd9964376b2b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:58.676666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e120f4-f04e-4efb-a0b7-cd9964376b2b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:58.676943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e120f4-f04e-4efb-a0b7-cd9964376b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:58.699634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-42e120f4-f04e-4efb-a0b7-cd9964376b2b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:58.701095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-42e120f4-f04e-4efb-a0b7-cd9964376b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:58.824252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 279/546] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:26:58.841791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957553f-e908-4d41-9b51-cf6e493461d7 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:58.846365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e120f4-f04e-4efb-a0b7-cd9964376b2b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:58.881199 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:58.922683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c477807-6bf1-4514-92b6-b0a50c737977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:58.923386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c477807-6bf1-4514-92b6-b0a50c737977 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:58.923672 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c477807-6bf1-4514-92b6-b0a50c737977 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:58.923955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c477807-6bf1-4514-92b6-b0a50c737977 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:58.924776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c477807-6bf1-4514-92b6-b0a50c737977 None admin] RBAC: Adding query filter params (name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:58.925117 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c477807-6bf1-4514-92b6-b0a50c737977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:58.926518 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c477807-6bf1-4514-92b6-b0a50c737977 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:58.949994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 268/547] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:26:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:58.955496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c477807-6bf1-4514-92b6-b0a50c737977 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:58.966452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e120f4-f04e-4efb-a0b7-cd9964376b2b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:58.993516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:59.003882 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4a696-799b-47c4-ba54-2e9beb6ff789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:59.004704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4a696-799b-47c4-ba54-2e9beb6ff789 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:59.005067 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4a696-799b-47c4-ba54-2e9beb6ff789 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:59.005349 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4a696-799b-47c4-ba54-2e9beb6ff789 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:59.021129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4a696-799b-47c4-ba54-2e9beb6ff789 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-e2e4a696-799b-47c4-ba54-2e9beb6ff789 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:26:59.029130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:59.031259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:59.032643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 280/548] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:26:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:26:59.043041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4a696-799b-47c4-ba54-2e9beb6ff789 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:59.097535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c477807-6bf1-4514-92b6-b0a50c737977 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:59.136189 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d3065-8ac6-4bab-bc78-02df59576d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:59.136900 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3065-8ac6-4bab-bc78-02df59576d34 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:59.137185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3065-8ac6-4bab-bc78-02df59576d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:59.137462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3065-8ac6-4bab-bc78-02df59576d34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:59.160186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d3065-8ac6-4bab-bc78-02df59576d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:59.161637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d3065-8ac6-4bab-bc78-02df59576d34 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:59.196864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4a696-799b-47c4-ba54-2e9beb6ff789 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-154d3065-8ac6-4bab-bc78-02df59576d34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:59.214217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:59.216248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:59.218580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:59.218580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:59.218580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:59.218580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:59.218580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 269/549] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:59.224517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d3065-8ac6-4bab-bc78-02df59576d34 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:59.234166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d7dc3-47b1-41bd-8eaf-bfd242a1a965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:59.235507 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d7dc3-47b1-41bd-8eaf-bfd242a1a965 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:59.236020 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d7dc3-47b1-41bd-8eaf-bfd242a1a965 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:59.236519 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d7dc3-47b1-41bd-8eaf-bfd242a1a965 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:59.237842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d7dc3-47b1-41bd-8eaf-bfd242a1a965 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:59.238741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d7dc3-47b1-41bd-8eaf-bfd242a1a965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:59.240784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d7dc3-47b1-41bd-8eaf-bfd242a1a965 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:59.251940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 281/550] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:26:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:59.261934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d7dc3-47b1-41bd-8eaf-bfd242a1a965 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:59.281781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:59.366319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:59.406498 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d3065-8ac6-4bab-bc78-02df59576d34 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:59.444208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d7dc3-47b1-41bd-8eaf-bfd242a1a965 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:59.475184 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c51fe-1d02-439f-826a-ffdc19fc404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:59.475855 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c51fe-1d02-439f-826a-ffdc19fc404d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:59.476135 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c51fe-1d02-439f-826a-ffdc19fc404d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:59.476406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c51fe-1d02-439f-826a-ffdc19fc404d None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:59.477135 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c51fe-1d02-439f-826a-ffdc19fc404d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:59.477440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c51fe-1d02-439f-826a-ffdc19fc404d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:59.478841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c51fe-1d02-439f-826a-ffdc19fc404d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:59.505693 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 270/551] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:59.509694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c51fe-1d02-439f-826a-ffdc19fc404d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:59.511348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666150b8-0a08-430a-9836-4b70e4f9a829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:59.512415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666150b8-0a08-430a-9836-4b70e4f9a829 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:59.512899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666150b8-0a08-430a-9836-4b70e4f9a829 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:59.513370 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666150b8-0a08-430a-9836-4b70e4f9a829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:59.534205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666150b8-0a08-430a-9836-4b70e4f9a829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:59.535989 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666150b8-0a08-430a-9836-4b70e4f9a829 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-666150b8-0a08-430a-9836-4b70e4f9a829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:26:59.573980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:59.574965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:59.575926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:26:59.575926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:26:59.575926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:26:59.575926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:26:59.575926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 282/552] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:59.582171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666150b8-0a08-430a-9836-4b70e4f9a829 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:59.615469 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:59.622788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c51fe-1d02-439f-826a-ffdc19fc404d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:59.657967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f770ba-9670-49d7-989f-9ba4b522a567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:59.658705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f770ba-9670-49d7-989f-9ba4b522a567 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:59.659035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f770ba-9670-49d7-989f-9ba4b522a567 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:59.659317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f770ba-9670-49d7-989f-9ba4b522a567 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:59.666623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f770ba-9670-49d7-989f-9ba4b522a567 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:59.668070 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f770ba-9670-49d7-989f-9ba4b522a567 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-09f770ba-9670-49d7-989f-9ba4b522a567 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:26:59.688989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:26:59.689944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:26:59.690863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:26:59.690863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:26:59.690863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:26:59.690863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:26:59.690863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:26:59.690863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:26:59.690863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:26:59.690863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:26:59.690863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:26:59.690863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:26:59.696038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 271/553] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:26:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:26:59.699230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f770ba-9670-49d7-989f-9ba4b522a567 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:59.782283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:59.788568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666150b8-0a08-430a-9836-4b70e4f9a829 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:59.831505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15054bf4-3e0f-47c3-91bb-ac18e2ae6783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:59.832762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15054bf4-3e0f-47c3-91bb-ac18e2ae6783 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:59.833388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15054bf4-3e0f-47c3-91bb-ac18e2ae6783 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:59.833867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15054bf4-3e0f-47c3-91bb-ac18e2ae6783 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:59.835441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15054bf4-3e0f-47c3-91bb-ac18e2ae6783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:59.835959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15054bf4-3e0f-47c3-91bb-ac18e2ae6783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:59.837835 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15054bf4-3e0f-47c3-91bb-ac18e2ae6783 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:59.855306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 283/554] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:26:59.857283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e36d17-9990-4a6e-9f4b-c6e124200aab None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:59.857736 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e36d17-9990-4a6e-9f4b-c6e124200aab None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:59.858195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e36d17-9990-4a6e-9f4b-c6e124200aab None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:59.859879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 284/555] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:26:59.860914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e36d17-9990-4a6e-9f4b-c6e124200aab None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:59.884957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f770ba-9670-49d7-989f-9ba4b522a567 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:59.930507 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ea5e2-6c40-4ce7-a289-84cd88a9cfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:59.931508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ea5e2-6c40-4ce7-a289-84cd88a9cfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:59.931988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ea5e2-6c40-4ce7-a289-84cd88a9cfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:59.932465 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ea5e2-6c40-4ce7-a289-84cd88a9cfe5 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:59.933640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ea5e2-6c40-4ce7-a289-84cd88a9cfe5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:26:59.934159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ea5e2-6c40-4ce7-a289-84cd88a9cfe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:59.935913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ea5e2-6c40-4ce7-a289-84cd88a9cfe5 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:59.955713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 272/556] 10.222.0.60 () {56 vars in 1135 bytes} [Mon Jul 25 12:26:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:26:59.957342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dce54-8049-4507-abdc-91cd788cc93d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:59.957822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dce54-8049-4507-abdc-91cd788cc93d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:59.958349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dce54-8049-4507-abdc-91cd788cc93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:59.977843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-977dce54-8049-4507-abdc-91cd788cc93d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:26:59.980058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-977dce54-8049-4507-abdc-91cd788cc93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:59.984034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e36d17-9990-4a6e-9f4b-c6e124200aab None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:59.991251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:00.020125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335037d-e41e-4449-aad9-55f4c984d4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:00.020927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335037d-e41e-4449-aad9-55f4c984d4af None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:00.021195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335037d-e41e-4449-aad9-55f4c984d4af None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:00.021470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335037d-e41e-4449-aad9-55f4c984d4af None admin] PATH_INFO: `/v3/users/nova` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:00.028757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335037d-e41e-4449-aad9-55f4c984d4af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:00.030227 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335037d-e41e-4449-aad9-55f4c984d4af None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c335037d-e41e-4449-aad9-55f4c984d4af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:00.045118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:00.046610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:00.047524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:00.047524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:00.047524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:00.047524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:00.047524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:00.047524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:00.047524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:00.047524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:00.047524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:00.047524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:00.050934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 285/557] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Jul 25 12:26:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:00.051767 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335037d-e41e-4449-aad9-55f4c984d4af None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:00.062693 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 273/558] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:00.063926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dce54-8049-4507-abdc-91cd788cc93d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:00.074809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:00.156094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335037d-e41e-4449-aad9-55f4c984d4af None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:00.179700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dce54-8049-4507-abdc-91cd788cc93d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:00.198042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77572d36-8317-4e41-8c20-f97003580e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:00.198700 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77572d36-8317-4e41-8c20-f97003580e1e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:00.198988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77572d36-8317-4e41-8c20-f97003580e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:00.199277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77572d36-8317-4e41-8c20-f97003580e1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:00.224699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d449547-dd9b-458f-927d-2614b24f8c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:00.225802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d449547-dd9b-458f-927d-2614b24f8c23 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:00.226341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d449547-dd9b-458f-927d-2614b24f8c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:00.226832 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d449547-dd9b-458f-927d-2614b24f8c23 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:00.228383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d449547-dd9b-458f-927d-2614b24f8c23 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:00.228955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d449547-dd9b-458f-927d-2614b24f8c23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:00.231536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d449547-dd9b-458f-927d-2614b24f8c23 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:00.251559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 274/559] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Jul 25 12:27:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:00.252701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d449547-dd9b-458f-927d-2614b24f8c23 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:00.261273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:00.282065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77572d36-8317-4e41-8c20-f97003580e1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:00.284199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77572d36-8317-4e41-8c20-f97003580e1e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:00.314929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:00.383349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d449547-dd9b-458f-927d-2614b24f8c23 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:00.420955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 286/560] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:27:00.422408 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77572d36-8317-4e41-8c20-f97003580e1e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:00.425974 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291be36-7668-4ea3-a9cb-b2a83a2e8329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:00.427007 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291be36-7668-4ea3-a9cb-b2a83a2e8329 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:00.427288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291be36-7668-4ea3-a9cb-b2a83a2e8329 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:00.427564 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291be36-7668-4ea3-a9cb-b2a83a2e8329 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:00.442600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291be36-7668-4ea3-a9cb-b2a83a2e8329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-e291be36-7668-4ea3-a9cb-b2a83a2e8329 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:00.452819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:00.453741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:00.454611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:00.454611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:00.454611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:00.454611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:00.454611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:00.454611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:00.454611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:00.454611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:00.454611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:00.454611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:00.454611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:00.454611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:00.457495 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 275/561] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:00.458550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291be36-7668-4ea3-a9cb-b2a83a2e8329 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:00.527182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:00.542186 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77572d36-8317-4e41-8c20-f97003580e1e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:00.563991 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:00.572297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291be36-7668-4ea3-a9cb-b2a83a2e8329 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:00.578943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a0337-575e-4572-b72f-1b32d0937732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:00.579737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a0337-575e-4572-b72f-1b32d0937732 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:00.580010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a0337-575e-4572-b72f-1b32d0937732 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:00.580282 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a0337-575e-4572-b72f-1b32d0937732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:00.591512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a0337-575e-4572-b72f-1b32d0937732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:00.592950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a0337-575e-4572-b72f-1b32d0937732 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:00.609244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57709ef0-1b2e-4bb2-90d1-ec05196c8982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:00.610051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57709ef0-1b2e-4bb2-90d1-ec05196c8982 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:00.610332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57709ef0-1b2e-4bb2-90d1-ec05196c8982 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:00.610602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57709ef0-1b2e-4bb2-90d1-ec05196c8982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-8f0a0337-575e-4572-b72f-1b32d0937732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:00.611399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:00.612239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:00.613148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:00.613148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:00.613148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:00.613148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:00.613148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 287/562] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:00.613426 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a0337-575e-4572-b72f-1b32d0937732 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:00.627290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57709ef0-1b2e-4bb2-90d1-ec05196c8982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:00.628709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57709ef0-1b2e-4bb2-90d1-ec05196c8982 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-57709ef0-1b2e-4bb2-90d1-ec05196c8982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:00.646295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:00.647337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:00.648409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:00.648409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:00.648409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:00.648409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:00.652590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 276/563] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:00.653894 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755242f7-8805-4822-b89e-5eaeadd6cda0 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:00.654200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755242f7-8805-4822-b89e-5eaeadd6cda0 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:00.654475 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755242f7-8805-4822-b89e-5eaeadd6cda0 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:00.655887 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 277/564] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:00.659178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755242f7-8805-4822-b89e-5eaeadd6cda0 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:00.722915 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a0337-575e-4572-b72f-1b32d0937732 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:00.756690 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62ac32-686f-4e33-a10a-69a09010fb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:00.757370 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62ac32-686f-4e33-a10a-69a09010fb6c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:00.757669 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62ac32-686f-4e33-a10a-69a09010fb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:00.757938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62ac32-686f-4e33-a10a-69a09010fb6c None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:00.758773 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62ac32-686f-4e33-a10a-69a09010fb6c None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:00.758995 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62ac32-686f-4e33-a10a-69a09010fb6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:00.760884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62ac32-686f-4e33-a10a-69a09010fb6c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:00.766978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755242f7-8805-4822-b89e-5eaeadd6cda0 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:00.769081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 288/565] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:27:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:00.770041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62ac32-686f-4e33-a10a-69a09010fb6c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:00.786881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:00.807330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db02ef6-8306-463f-b568-da9932c877a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:00.807983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db02ef6-8306-463f-b568-da9932c877a8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:00.808261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db02ef6-8306-463f-b568-da9932c877a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:00.808529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db02ef6-8306-463f-b568-da9932c877a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:00.809247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db02ef6-8306-463f-b568-da9932c877a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:00.809547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db02ef6-8306-463f-b568-da9932c877a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:00.810950 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db02ef6-8306-463f-b568-da9932c877a8 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:00.811091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:00.823822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 278/566] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:00.825292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a79da-0ce7-4d00-9efc-a84ec63f792e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:00.825412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a79da-0ce7-4d00-9efc-a84ec63f792e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:00.825661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a79da-0ce7-4d00-9efc-a84ec63f792e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:00.844644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-426a79da-0ce7-4d00-9efc-a84ec63f792e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:00.845831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-426a79da-0ce7-4d00-9efc-a84ec63f792e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:00.876987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62ac32-686f-4e33-a10a-69a09010fb6c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:00.909655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b24573-145b-44e9-ba8d-87e5b0c094bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:00.910338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b24573-145b-44e9-ba8d-87e5b0c094bd None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:00.910615 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b24573-145b-44e9-ba8d-87e5b0c094bd None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:00.910898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b24573-145b-44e9-ba8d-87e5b0c094bd None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:00.911547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 279/567] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:00.911983 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b24573-145b-44e9-ba8d-87e5b0c094bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:00.911983 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b24573-145b-44e9-ba8d-87e5b0c094bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:00.912374 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a79da-0ce7-4d00-9efc-a84ec63f792e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:00.913240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b24573-145b-44e9-ba8d-87e5b0c094bd None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:00.931910 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 289/568] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:00.933288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f30eb-c144-475b-a9b0-ef8c536bbdae None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:00.933575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f30eb-c144-475b-a9b0-ef8c536bbdae None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:00.933853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f30eb-c144-475b-a9b0-ef8c536bbdae None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:00.935093 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 290/569] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:00.935774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f30eb-c144-475b-a9b0-ef8c536bbdae None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:01.013259 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a79da-0ce7-4d00-9efc-a84ec63f792e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:01.015471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:01.028641 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:01.036561 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f30eb-c144-475b-a9b0-ef8c536bbdae None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:01.046842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c0880-7c3b-4a4e-a13f-cc17e8b7dad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:01.047622 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c0880-7c3b-4a4e-a13f-cc17e8b7dad2 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.047901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c0880-7c3b-4a4e-a13f-cc17e8b7dad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.048176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c0880-7c3b-4a4e-a13f-cc17e8b7dad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.058500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c0880-7c3b-4a4e-a13f-cc17e8b7dad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:01.059935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c0880-7c3b-4a4e-a13f-cc17e8b7dad2 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:01.069617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e034ec5-3cf5-4272-a440-ca6343684779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:01.070424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e034ec5-3cf5-4272-a440-ca6343684779 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.070755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e034ec5-3cf5-4272-a440-ca6343684779 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.071040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e034ec5-3cf5-4272-a440-ca6343684779 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-2b6c0880-7c3b-4a4e-a13f-cc17e8b7dad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:01.077403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:01.078430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:01.079483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:01.079483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:01.079483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:01.079483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:01.079727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e034ec5-3cf5-4272-a440-ca6343684779 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:01.079727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e034ec5-3cf5-4272-a440-ca6343684779 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:01.082236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 280/570] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:01.083048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c0880-7c3b-4a4e-a13f-cc17e8b7dad2 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-2e034ec5-3cf5-4272-a440-ca6343684779 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:01.093547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:01.094629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:01.095628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:01.095628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:01.095628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:01.095628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:01.095628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:01.095628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:01.095628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:01.095628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:01.095628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 291/571] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:01.095628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e034ec5-3cf5-4272-a440-ca6343684779 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:01.182631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c0880-7c3b-4a4e-a13f-cc17e8b7dad2 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:01.201869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e034ec5-3cf5-4272-a440-ca6343684779 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:01.216972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be74d2-225c-4199-a5df-c2df1b757cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:01.217765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be74d2-225c-4199-a5df-c2df1b757cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.218073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be74d2-225c-4199-a5df-c2df1b757cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be74d2-225c-4199-a5df-c2df1b757cb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.228384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be74d2-225c-4199-a5df-c2df1b757cb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:01.235436 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7e650-0a2d-473f-9aca-4ff1aa53f95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:01.236211 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7e650-0a2d-473f-9aca-4ff1aa53f95a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.236491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7e650-0a2d-473f-9aca-4ff1aa53f95a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.236763 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7e650-0a2d-473f-9aca-4ff1aa53f95a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-84be74d2-225c-4199-a5df-c2df1b757cb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:01.237293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:01.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:01.239445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 281/572] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:01.240288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd9338-0d4b-4054-9fdd-e72df51beb1f None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.240288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd9338-0d4b-4054-9fdd-e72df51beb1f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.240288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd9338-0d4b-4054-9fdd-e72df51beb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.251807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7e650-0a2d-473f-9aca-4ff1aa53f95a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:01.253276 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7e650-0a2d-473f-9aca-4ff1aa53f95a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:01.258341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-aadd9338-0d4b-4054-9fdd-e72df51beb1f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:01.259516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-aadd9338-0d4b-4054-9fdd-e72df51beb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-9ab7e650-0a2d-473f-9aca-4ff1aa53f95a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:01.267872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:01.268909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:01.270353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:01.270353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:01.270353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:01.270353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:01.270353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:01.270353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:01.270353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:01.270353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:01.270353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 292/573] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:01.270353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7e650-0a2d-473f-9aca-4ff1aa53f95a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:01.270992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:01.285228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:01.341947 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 282/574] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:01.341947 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd9338-0d4b-4054-9fdd-e72df51beb1f None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:01.368409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7e650-0a2d-473f-9aca-4ff1aa53f95a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:01.401584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d137b8-d019-454a-9e84-8a4a35f812ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:01.402275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d137b8-d019-454a-9e84-8a4a35f812ed None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.402553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d137b8-d019-454a-9e84-8a4a35f812ed None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.402824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d137b8-d019-454a-9e84-8a4a35f812ed None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.403535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d137b8-d019-454a-9e84-8a4a35f812ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:01.403839 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d137b8-d019-454a-9e84-8a4a35f812ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:01.405227 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d137b8-d019-454a-9e84-8a4a35f812ed None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:01.416109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 293/575] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:01.417245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d137b8-d019-454a-9e84-8a4a35f812ed None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:01.427629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd9338-0d4b-4054-9fdd-e72df51beb1f None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:01.460540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1ae96-82f2-407e-aa9d-7f2fe7544e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:01.461193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1ae96-82f2-407e-aa9d-7f2fe7544e3e None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.461473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1ae96-82f2-407e-aa9d-7f2fe7544e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.461753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1ae96-82f2-407e-aa9d-7f2fe7544e3e None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.462574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1ae96-82f2-407e-aa9d-7f2fe7544e3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:01.462884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1ae96-82f2-407e-aa9d-7f2fe7544e3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:01.464385 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1ae96-82f2-407e-aa9d-7f2fe7544e3e None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:01.475691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 283/576] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:01.476289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1ae96-82f2-407e-aa9d-7f2fe7544e3e None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:01.485472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:01.491769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:01.517979 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d137b8-d019-454a-9e84-8a4a35f812ed None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:01.550558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ad183-6090-4ddb-8c7f-bf55a5a70151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:01.551220 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ad183-6090-4ddb-8c7f-bf55a5a70151 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.551503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ad183-6090-4ddb-8c7f-bf55a5a70151 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.551782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ad183-6090-4ddb-8c7f-bf55a5a70151 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.552522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ad183-6090-4ddb-8c7f-bf55a5a70151 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:01.552829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ad183-6090-4ddb-8c7f-bf55a5a70151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:01.554226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ad183-6090-4ddb-8c7f-bf55a5a70151 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:01.562609 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 294/577] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:27:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:01.563389 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ad183-6090-4ddb-8c7f-bf55a5a70151 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:01.575336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1ae96-82f2-407e-aa9d-7f2fe7544e3e None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:01.608596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57947b-7ffe-4c50-8d28-67183961e488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:01.609251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57947b-7ffe-4c50-8d28-67183961e488 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.609526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57947b-7ffe-4c50-8d28-67183961e488 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.609795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57947b-7ffe-4c50-8d28-67183961e488 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.610604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57947b-7ffe-4c50-8d28-67183961e488 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:01.610905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57947b-7ffe-4c50-8d28-67183961e488 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:01.612302 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57947b-7ffe-4c50-8d28-67183961e488 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:01.624276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 284/578] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:01.625041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57947b-7ffe-4c50-8d28-67183961e488 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:01.667329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ad183-6090-4ddb-8c7f-bf55a5a70151 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:01.686782 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:01.700459 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7a956-8c0c-4b13-a835-a7e574283df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:01.701244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7a956-8c0c-4b13-a835-a7e574283df3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.701529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7a956-8c0c-4b13-a835-a7e574283df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.701834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7a956-8c0c-4b13-a835-a7e574283df3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.704879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:01.713677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7a956-8c0c-4b13-a835-a7e574283df3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:01.720628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57947b-7ffe-4c50-8d28-67183961e488 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-41e7a956-8c0c-4b13-a835-a7e574283df3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:01.721453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:01.722540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 295/579] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:01.723527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7a956-8c0c-4b13-a835-a7e574283df3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:01.755546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151e057-43a8-4bae-b11d-516ea1a26bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:01.756339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151e057-43a8-4bae-b11d-516ea1a26bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.756620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151e057-43a8-4bae-b11d-516ea1a26bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.756895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151e057-43a8-4bae-b11d-516ea1a26bd9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.765656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151e057-43a8-4bae-b11d-516ea1a26bd9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:01.767246 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151e057-43a8-4bae-b11d-516ea1a26bd9 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-8151e057-43a8-4bae-b11d-516ea1a26bd9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:01.786795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:01.787845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:01.788771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:01.788771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:01.788771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:01.788771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:01.788771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:01.788771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:01.788771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:01.788771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:01.788771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:27:01.788771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:01.788771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 285/580] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:01.788771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151e057-43a8-4bae-b11d-516ea1a26bd9 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:01.826608 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7a956-8c0c-4b13-a835-a7e574283df3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:01.860158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86b531-a332-41a3-857d-8bf6ba1b6e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:01.861029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86b531-a332-41a3-857d-8bf6ba1b6e66 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.861307 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86b531-a332-41a3-857d-8bf6ba1b6e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.861580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86b531-a332-41a3-857d-8bf6ba1b6e66 None admin] PATH_INFO: `/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/ccf2dedaf7a449d2b51b9e9f3b420091/roles/d9c5c1adb796486d83edcd6e728a5de9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.875170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86b531-a332-41a3-857d-8bf6ba1b6e66 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff5a41f8ca6438295c512792d2022e4, user_id=ccf2dedaf7a449d2b51b9e9f3b420091, role_id=d9c5c1adb796486d83edcd6e728a5de9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:01.876600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86b531-a332-41a3-857d-8bf6ba1b6e66 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:01.887802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151e057-43a8-4bae-b11d-516ea1a26bd9 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:01.888997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 296/581] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:27:01 2022] PUT /identity/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/ccf2dedaf7a449d2b51b9e9f3b420091/roles/d9c5c1adb796486d83edcd6e728a5de9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:27:01.889775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86b531-a332-41a3-857d-8bf6ba1b6e66 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:01.918724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:01.933559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c090c39-af26-4e54-907b-1d3d8867a6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:01.934507 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c090c39-af26-4e54-907b-1d3d8867a6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:01.934799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c090c39-af26-4e54-907b-1d3d8867a6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:01.935206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c090c39-af26-4e54-907b-1d3d8867a6a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:01.944098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:01.946905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c090c39-af26-4e54-907b-1d3d8867a6a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:01.948871 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c090c39-af26-4e54-907b-1d3d8867a6a7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-4c090c39-af26-4e54-907b-1d3d8867a6a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:01.968598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:01.969573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:01.971052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:01.971052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:01.971052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:01.971052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:01.971052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 286/582] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:01.971052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c090c39-af26-4e54-907b-1d3d8867a6a7 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:02.014226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86b531-a332-41a3-857d-8bf6ba1b6e66 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:02.056416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec57ffd-a0e3-46e2-bc25-95854dc8ad1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:02.057111 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec57ffd-a0e3-46e2-bc25-95854dc8ad1b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:02.057401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec57ffd-a0e3-46e2-bc25-95854dc8ad1b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:02.057679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec57ffd-a0e3-46e2-bc25-95854dc8ad1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:02.058588 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec57ffd-a0e3-46e2-bc25-95854dc8ad1b None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=e56b0d48e08c4f778a7c509c10edc871) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:02.058904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec57ffd-a0e3-46e2-bc25-95854dc8ad1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:02.060359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec57ffd-a0e3-46e2-bc25-95854dc8ad1b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:02.066517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 297/583] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:01 2022] GET /identity/v3/role_assignments?user.id=e56b0d48e08c4f778a7c509c10edc871&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:02.067600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec57ffd-a0e3-46e2-bc25-95854dc8ad1b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:02.092395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c090c39-af26-4e54-907b-1d3d8867a6a7 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:02.134784 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8e348-054f-43a4-80c4-419e4905afff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:02.135120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8e348-054f-43a4-80c4-419e4905afff None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:02.136027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8e348-054f-43a4-80c4-419e4905afff None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:02.136303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8e348-054f-43a4-80c4-419e4905afff None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:02.137079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8e348-054f-43a4-80c4-419e4905afff None admin] RBAC: Adding query filter params (name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:02.137386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8e348-054f-43a4-80c4-419e4905afff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:02.140002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8e348-054f-43a4-80c4-419e4905afff None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:02.154085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 287/584] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:27:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:02.154085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8e348-054f-43a4-80c4-419e4905afff None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:02.205759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:02.242042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec57ffd-a0e3-46e2-bc25-95854dc8ad1b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:02.263544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:02.277776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8e348-054f-43a4-80c4-419e4905afff None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:02.285662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66a26e-c207-4b25-8f85-ec5f15c0ab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:02.286771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66a26e-c207-4b25-8f85-ec5f15c0ab5e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:02.286771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66a26e-c207-4b25-8f85-ec5f15c0ab5e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:02.289874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66a26e-c207-4b25-8f85-ec5f15c0ab5e None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:02.289874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66a26e-c207-4b25-8f85-ec5f15c0ab5e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:02.289874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66a26e-c207-4b25-8f85-ec5f15c0ab5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:02.289874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66a26e-c207-4b25-8f85-ec5f15c0ab5e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:02.318069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 298/585] 10.222.0.60 () {56 vars in 1137 bytes} [Mon Jul 25 12:27:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:02.319549 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66a26e-c207-4b25-8f85-ec5f15c0ab5e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:02.323451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bb1a5-a363-4c15-aa7b-edb93d337247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:02.324763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bb1a5-a363-4c15-aa7b-edb93d337247 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:02.324763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bb1a5-a363-4c15-aa7b-edb93d337247 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:02.324763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bb1a5-a363-4c15-aa7b-edb93d337247 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:02.325399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bb1a5-a363-4c15-aa7b-edb93d337247 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:02.325711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bb1a5-a363-4c15-aa7b-edb93d337247 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:02.327229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bb1a5-a363-4c15-aa7b-edb93d337247 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:02.344514 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 288/586] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:02.344514 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bb1a5-a363-4c15-aa7b-edb93d337247 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:02.434344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66a26e-c207-4b25-8f85-ec5f15c0ab5e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:02.466338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:02.469101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bb1a5-a363-4c15-aa7b-edb93d337247 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:02.470261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ac9c-b370-4343-b85d-058606647326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:02.473237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ac9c-b370-4343-b85d-058606647326 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:02.473237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ac9c-b370-4343-b85d-058606647326 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:02.473237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ac9c-b370-4343-b85d-058606647326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:02.484236 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316ac9c-b370-4343-b85d-058606647326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:02.486133 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316ac9c-b370-4343-b85d-058606647326 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b316ac9c-b370-4343-b85d-058606647326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:02.506508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:02.507603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:02.508691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:02.508691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:02.508691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:02.508691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:02.510206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704160e-7a8d-4ab0-87c9-165354cb7704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:02.515020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704160e-7a8d-4ab0-87c9-165354cb7704 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:02.515020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704160e-7a8d-4ab0-87c9-165354cb7704 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:02.515020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704160e-7a8d-4ab0-87c9-165354cb7704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:02.515303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 299/587] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:02.515303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ac9c-b370-4343-b85d-058606647326 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:02.528797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704160e-7a8d-4ab0-87c9-165354cb7704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:02.530707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704160e-7a8d-4ab0-87c9-165354cb7704 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:02.547810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-3704160e-7a8d-4ab0-87c9-165354cb7704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:02.553038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:02.554104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:02.555161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:02.555161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:02.555161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:02.555161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:02.555161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 289/588] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:02.555502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704160e-7a8d-4ab0-87c9-165354cb7704 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:02.649207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ac9c-b370-4343-b85d-058606647326 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:02.691153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704160e-7a8d-4ab0-87c9-165354cb7704 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:02.694873 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687d634-7099-4cfc-9fe2-68b13bd07f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:02.699143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687d634-7099-4cfc-9fe2-68b13bd07f5c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:02.699570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687d634-7099-4cfc-9fe2-68b13bd07f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:02.699988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687d634-7099-4cfc-9fe2-68b13bd07f5c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:02.710206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687d634-7099-4cfc-9fe2-68b13bd07f5c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:02.712455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687d634-7099-4cfc-9fe2-68b13bd07f5c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:02.730351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bca897-7792-4da6-977b-bee4364998c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:02.731491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bca897-7792-4da6-977b-bee4364998c2 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:02.731875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bca897-7792-4da6-977b-bee4364998c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-d687d634-7099-4cfc-9fe2-68b13bd07f5c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:02.732126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:02.733195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:02.734446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:02.734446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:02.734446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:02.734446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:02.734446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:02.734446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:02.734446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:02.734446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:02.734446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 12:27:02.734446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:02.735072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bca897-7792-4da6-977b-bee4364998c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:02.735652 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bca897-7792-4da6-977b-bee4364998c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:02.736067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bca897-7792-4da6-977b-bee4364998c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:02.737029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 300/589] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:02.737755 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bca897-7792-4da6-977b-bee4364998c2 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:02.738224 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687d634-7099-4cfc-9fe2-68b13bd07f5c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:02.753326 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 290/590] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:02.755064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bca897-7792-4da6-977b-bee4364998c2 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:02.775385 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:02.838918 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:02.860869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687d634-7099-4cfc-9fe2-68b13bd07f5c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:02.869404 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bca897-7792-4da6-977b-bee4364998c2 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:02.898234 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37d5f2-73df-4611-a6b2-19b52d752237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:02.899291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37d5f2-73df-4611-a6b2-19b52d752237 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:02.899776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37d5f2-73df-4611-a6b2-19b52d752237 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:02.900264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37d5f2-73df-4611-a6b2-19b52d752237 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:02.901229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37d5f2-73df-4611-a6b2-19b52d752237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:02.901742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37d5f2-73df-4611-a6b2-19b52d752237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:02.903316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37d5f2-73df-4611-a6b2-19b52d752237 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:02.905135 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb0ed3-1e0d-4c0a-a707-bd936370906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:02.905850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb0ed3-1e0d-4c0a-a707-bd936370906f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:02.906170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb0ed3-1e0d-4c0a-a707-bd936370906f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:02.906447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb0ed3-1e0d-4c0a-a707-bd936370906f None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:02.907283 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb0ed3-1e0d-4c0a-a707-bd936370906f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:02.907587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb0ed3-1e0d-4c0a-a707-bd936370906f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:02.909050 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb0ed3-1e0d-4c0a-a707-bd936370906f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:02.917810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 301/591] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:02.919046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37d5f2-73df-4611-a6b2-19b52d752237 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:02.929227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 291/592] 10.222.0.60 () {56 vars in 1137 bytes} [Mon Jul 25 12:27:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:02.930088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb0ed3-1e0d-4c0a-a707-bd936370906f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:03.002131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:03.027531 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37d5f2-73df-4611-a6b2-19b52d752237 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:03.035524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb0ed3-1e0d-4c0a-a707-bd936370906f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:03.064395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e50c5-ec0f-49f3-b9da-fda42020bf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:03.065559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e50c5-ec0f-49f3-b9da-fda42020bf79 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:03.066114 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e50c5-ec0f-49f3-b9da-fda42020bf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:03.066614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e50c5-ec0f-49f3-b9da-fda42020bf79 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:03.071453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f1ad2-8572-4630-834e-6bf31ceb1094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:03.072240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f1ad2-8572-4630-834e-6bf31ceb1094 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:03.072518 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f1ad2-8572-4630-834e-6bf31ceb1094 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:03.072822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f1ad2-8572-4630-834e-6bf31ceb1094 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:03.075764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e50c5-ec0f-49f3-b9da-fda42020bf79 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:03.077435 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e50c5-ec0f-49f3-b9da-fda42020bf79 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:03.080270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f1ad2-8572-4630-834e-6bf31ceb1094 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:03.080783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:03.082134 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f1ad2-8572-4630-834e-6bf31ceb1094 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-158e50c5-ec0f-49f3-b9da-fda42020bf79 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:03.094663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:03.095617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:03.096560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:03.096560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:03.096560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:03.096560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:03.096560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:03.096560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:03.096560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:03.096560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:03.096560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 12:27:03.096560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-014f1ad2-8572-4630-834e-6bf31ceb1094 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:03.097081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:03.098574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:03.099458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:03.099458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:03.099458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:03.099458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:03.099458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:03.099458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:03.099458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:03.099458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:03.099458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 292/593] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:03.099458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f1ad2-8572-4630-834e-6bf31ceb1094 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:03.101871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 302/594] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:03.110876 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e50c5-ec0f-49f3-b9da-fda42020bf79 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:03.210802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f1ad2-8572-4630-834e-6bf31ceb1094 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:03.241287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:03.249541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3fadd-eaed-4688-9554-e722a329603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:03.250441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3fadd-eaed-4688-9554-e722a329603f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:03.250723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3fadd-eaed-4688-9554-e722a329603f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:03.251041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3fadd-eaed-4688-9554-e722a329603f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:03.252346 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e50c5-ec0f-49f3-b9da-fda42020bf79 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:03.262880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3fadd-eaed-4688-9554-e722a329603f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:03.264397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3fadd-eaed-4688-9554-e722a329603f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-2db3fadd-eaed-4688-9554-e722a329603f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:03.282422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:03.283517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:03.284576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:03.284576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:03.284576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:03.284576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:03.284576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 293/595] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:03.285044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3fadd-eaed-4688-9554-e722a329603f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:03.297758 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095182ac-3214-45e4-ac3b-a2db748d274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:03.299403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095182ac-3214-45e4-ac3b-a2db748d274a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:03.299872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095182ac-3214-45e4-ac3b-a2db748d274a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:03.300327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095182ac-3214-45e4-ac3b-a2db748d274a None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:03.301625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095182ac-3214-45e4-ac3b-a2db748d274a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:03.302550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095182ac-3214-45e4-ac3b-a2db748d274a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:03.307334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095182ac-3214-45e4-ac3b-a2db748d274a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:03.331430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 303/596] 10.222.0.60 () {56 vars in 1139 bytes} [Mon Jul 25 12:27:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:03.333039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095182ac-3214-45e4-ac3b-a2db748d274a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:03.405499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:03.410546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3fadd-eaed-4688-9554-e722a329603f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:03.446007 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572516b-bdc9-4078-9cb2-e951744cb2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:03.446729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572516b-bdc9-4078-9cb2-e951744cb2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:03.447105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572516b-bdc9-4078-9cb2-e951744cb2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:03.447708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572516b-bdc9-4078-9cb2-e951744cb2b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:03.448395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572516b-bdc9-4078-9cb2-e951744cb2b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:03.448801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572516b-bdc9-4078-9cb2-e951744cb2b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:03.450365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572516b-bdc9-4078-9cb2-e951744cb2b4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:03.462617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095182ac-3214-45e4-ac3b-a2db748d274a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:03.463252 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 294/597] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:03.464673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572516b-bdc9-4078-9cb2-e951744cb2b4 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:03.509805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:03.515951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b22cb5-6428-444b-b1f3-4762f2865f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:03.517147 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b22cb5-6428-444b-b1f3-4762f2865f0c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:03.517659 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b22cb5-6428-444b-b1f3-4762f2865f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:03.518287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b22cb5-6428-444b-b1f3-4762f2865f0c None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:03.519544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b22cb5-6428-444b-b1f3-4762f2865f0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:03.520091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b22cb5-6428-444b-b1f3-4762f2865f0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:03.522128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b22cb5-6428-444b-b1f3-4762f2865f0c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:03.538885 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 304/598] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:03.540911 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b22cb5-6428-444b-b1f3-4762f2865f0c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:03.570421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572516b-bdc9-4078-9cb2-e951744cb2b4 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:03.606542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3038ad6-35d5-4bd6-bcd9-a3bb0cf7edcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:03.607342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3038ad6-35d5-4bd6-bcd9-a3bb0cf7edcb None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:03.607616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3038ad6-35d5-4bd6-bcd9-a3bb0cf7edcb None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:03.607884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3038ad6-35d5-4bd6-bcd9-a3bb0cf7edcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:03.618769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3038ad6-35d5-4bd6-bcd9-a3bb0cf7edcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:03.620246 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3038ad6-35d5-4bd6-bcd9-a3bb0cf7edcb None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-a3038ad6-35d5-4bd6-bcd9-a3bb0cf7edcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:03.640195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:03.641131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:03.642676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:03.642676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:03.642676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:03.642676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:03.642676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 295/599] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:03.642676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3038ad6-35d5-4bd6-bcd9-a3bb0cf7edcb None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:03.680937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b22cb5-6428-444b-b1f3-4762f2865f0c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:03.704958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:03.730962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04d096-1259-45b6-80b7-21d02d8e32c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:03.732703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04d096-1259-45b6-80b7-21d02d8e32c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:03.733544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04d096-1259-45b6-80b7-21d02d8e32c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:03.734063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04d096-1259-45b6-80b7-21d02d8e32c3 None admin] PATH_INFO: `/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/eb4c5027951c4c55bd826943cec15309/roles/d9c5c1adb796486d83edcd6e728a5de9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:03.742435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:03.750408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3038ad6-35d5-4bd6-bcd9-a3bb0cf7edcb None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:03.756272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04d096-1259-45b6-80b7-21d02d8e32c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff5a41f8ca6438295c512792d2022e4, user_id=eb4c5027951c4c55bd826943cec15309, role_id=d9c5c1adb796486d83edcd6e728a5de9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:03.758546 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04d096-1259-45b6-80b7-21d02d8e32c3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:03.778164 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 305/600] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:27:03 2022] PUT /identity/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/eb4c5027951c4c55bd826943cec15309/roles/d9c5c1adb796486d83edcd6e728a5de9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:27:03.780055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04d096-1259-45b6-80b7-21d02d8e32c3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:03.788606 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7962d-110b-499f-9939-b2b11357009b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:03.789445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7962d-110b-499f-9939-b2b11357009b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:03.789712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7962d-110b-499f-9939-b2b11357009b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:03.790043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7962d-110b-499f-9939-b2b11357009b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:03.797806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7962d-110b-499f-9939-b2b11357009b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:03.799140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7962d-110b-499f-9939-b2b11357009b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-c3d7962d-110b-499f-9939-b2b11357009b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:03.813899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:03.814934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:03.816287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:03.816287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:03.816287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:03.816287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:03.816287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:03.816287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:03.816287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:03.816287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:03.816287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 296/601] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:03.821019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7962d-110b-499f-9939-b2b11357009b None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:03.935179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04d096-1259-45b6-80b7-21d02d8e32c3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:03.936651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7962d-110b-499f-9939-b2b11357009b None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:03.974916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b561d6-5b87-45c9-8502-cc9ee795cc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:03.975500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b561d6-5b87-45c9-8502-cc9ee795cc2b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:03.975778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b561d6-5b87-45c9-8502-cc9ee795cc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:03.976049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b561d6-5b87-45c9-8502-cc9ee795cc2b None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:03.976846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b561d6-5b87-45c9-8502-cc9ee795cc2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:03.977156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b561d6-5b87-45c9-8502-cc9ee795cc2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:03.978649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b561d6-5b87-45c9-8502-cc9ee795cc2b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:03.979172 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0cffc-5263-4838-9e09-f9f2182938be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:03.980098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0cffc-5263-4838-9e09-f9f2182938be None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:03.980597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0cffc-5263-4838-9e09-f9f2182938be None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:03.981354 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0cffc-5263-4838-9e09-f9f2182938be None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:03.982517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0cffc-5263-4838-9e09-f9f2182938be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:03.982967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0cffc-5263-4838-9e09-f9f2182938be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:03.984485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0cffc-5263-4838-9e09-f9f2182938be None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:03.997551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 297/602] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:03.998554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 306/603] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:03.999469 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535fb4a-7dc9-457e-a6fa-8f834166f4be None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:03.999765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535fb4a-7dc9-457e-a6fa-8f834166f4be None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.000047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535fb4a-7dc9-457e-a6fa-8f834166f4be None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.001480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 298/604] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:04.003497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0cffc-5263-4838-9e09-f9f2182938be None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:04.006254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535fb4a-7dc9-457e-a6fa-8f834166f4be None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:04.023218 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:04.071849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:04.116161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535fb4a-7dc9-457e-a6fa-8f834166f4be None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:04.120083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0cffc-5263-4838-9e09-f9f2182938be None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:04.152811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad4fd6-d46b-492a-b4bc-5ed1f9e8f402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:04.153625 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad4fd6-d46b-492a-b4bc-5ed1f9e8f402 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:04.153924 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad4fd6-d46b-492a-b4bc-5ed1f9e8f402 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.157459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad4fd6-d46b-492a-b4bc-5ed1f9e8f402 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.161232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b75b1-5a28-4ec9-8ac6-6aa10bf60cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:04.162158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b75b1-5a28-4ec9-8ac6-6aa10bf60cc1 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:04.162441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b75b1-5a28-4ec9-8ac6-6aa10bf60cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.162715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b75b1-5a28-4ec9-8ac6-6aa10bf60cc1 None admin] PATH_INFO: `/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/6b07dcd8c0de4d1694d4363520726df2/roles/d9c5c1adb796486d83edcd6e728a5de9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.164898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad4fd6-d46b-492a-b4bc-5ed1f9e8f402 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:04.169421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad4fd6-d46b-492a-b4bc-5ed1f9e8f402 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:04.182815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b75b1-5a28-4ec9-8ac6-6aa10bf60cc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff5a41f8ca6438295c512792d2022e4, user_id=6b07dcd8c0de4d1694d4363520726df2, role_id=d9c5c1adb796486d83edcd6e728a5de9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:04.184288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b75b1-5a28-4ec9-8ac6-6aa10bf60cc1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-a1ad4fd6-d46b-492a-b4bc-5ed1f9e8f402 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:04.189258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:04.190480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:04.191543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:04.191543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:04.191543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:04.191543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:04.191543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:04.191543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:04.191543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:04.191543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:04.191543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 299/605] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:04.197453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2942d0-a3e8-4cfe-93d6-3f3897711c47 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:04.198399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2942d0-a3e8-4cfe-93d6-3f3897711c47 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.199007 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2942d0-a3e8-4cfe-93d6-3f3897711c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.205111 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 307/606] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:27:04 2022] PUT /identity/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/6b07dcd8c0de4d1694d4363520726df2/roles/d9c5c1adb796486d83edcd6e728a5de9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:27:04.206265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b75b1-5a28-4ec9-8ac6-6aa10bf60cc1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:04.221141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-de2942d0-a3e8-4cfe-93d6-3f3897711c47 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:04.222639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-de2942d0-a3e8-4cfe-93d6-3f3897711c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:04.318646 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 300/607] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:04.323954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:04.346081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2942d0-a3e8-4cfe-93d6-3f3897711c47 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:04.350180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:04.351323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b75b1-5a28-4ec9-8ac6-6aa10bf60cc1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:04.399425 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2464e1-f9a1-411f-a81d-91bd13a29f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:04.400399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2464e1-f9a1-411f-a81d-91bd13a29f38 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:04.400864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2464e1-f9a1-411f-a81d-91bd13a29f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.401865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2464e1-f9a1-411f-a81d-91bd13a29f38 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.402868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2464e1-f9a1-411f-a81d-91bd13a29f38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:04.403667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2464e1-f9a1-411f-a81d-91bd13a29f38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:04.406053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2464e1-f9a1-411f-a81d-91bd13a29f38 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:04.423036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 308/608] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:04.424801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f2fe0-8114-45ee-a195-6864245261f1 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:04.425239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f2fe0-8114-45ee-a195-6864245261f1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.425660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f2fe0-8114-45ee-a195-6864245261f1 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.427562 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 309/609] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:04.429119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f2fe0-8114-45ee-a195-6864245261f1 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:04.475452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2942d0-a3e8-4cfe-93d6-3f3897711c47 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:04.524036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26199b14-2e95-4c72-aee3-c7f15358e904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:04.525085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26199b14-2e95-4c72-aee3-c7f15358e904 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:04.525561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26199b14-2e95-4c72-aee3-c7f15358e904 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.526062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26199b14-2e95-4c72-aee3-c7f15358e904 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.541005 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26199b14-2e95-4c72-aee3-c7f15358e904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-26199b14-2e95-4c72-aee3-c7f15358e904 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:04.553844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:04.555085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:04.556019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:04.556019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:04.556019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:04.556019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:04.556019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:04.556019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:04.556019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:04.556019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:04.556019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:04.556019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:04.556019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:04.556019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:04.556727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f2fe0-8114-45ee-a195-6864245261f1 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:04.560148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 301/610] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:04.561612 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246b9f4-9eed-492a-8cf4-829b60764e0e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:04.562107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246b9f4-9eed-492a-8cf4-829b60764e0e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.562526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246b9f4-9eed-492a-8cf4-829b60764e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.587108 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-7246b9f4-9eed-492a-8cf4-829b60764e0e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:04.589493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7246b9f4-9eed-492a-8cf4-829b60764e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:04.593280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3569b-6fcb-4924-8f5c-9d366df2c8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:04.594335 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3569b-6fcb-4924-8f5c-9d366df2c8e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:04.594733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3569b-6fcb-4924-8f5c-9d366df2c8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.595109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3569b-6fcb-4924-8f5c-9d366df2c8e3 None admin] PATH_INFO: `/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/375d321d6de2494794254e50151c02c4/roles/d9c5c1adb796486d83edcd6e728a5de9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.604616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:04.612075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3569b-6fcb-4924-8f5c-9d366df2c8e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff5a41f8ca6438295c512792d2022e4, user_id=375d321d6de2494794254e50151c02c4, role_id=d9c5c1adb796486d83edcd6e728a5de9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:04.614072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3569b-6fcb-4924-8f5c-9d366df2c8e3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:04.629802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 310/611] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:27:04 2022] PUT /identity/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/375d321d6de2494794254e50151c02c4/roles/d9c5c1adb796486d83edcd6e728a5de9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:27:04.631033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3569b-6fcb-4924-8f5c-9d366df2c8e3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:04.669521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:04.680524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 302/612] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:04.705373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246b9f4-9eed-492a-8cf4-829b60764e0e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:04.746648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3569b-6fcb-4924-8f5c-9d366df2c8e3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:04.784407 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13574460-49ee-4cc2-83f9-a977fca4427b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:04.785248 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13574460-49ee-4cc2-83f9-a977fca4427b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:04.786340 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13574460-49ee-4cc2-83f9-a977fca4427b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.786340 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13574460-49ee-4cc2-83f9-a977fca4427b None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.786938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13574460-49ee-4cc2-83f9-a977fca4427b None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:04.787369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13574460-49ee-4cc2-83f9-a977fca4427b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:04.788902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13574460-49ee-4cc2-83f9-a977fca4427b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:04.799520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 311/613] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:27:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:04.809945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13574460-49ee-4cc2-83f9-a977fca4427b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:04.822708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246b9f4-9eed-492a-8cf4-829b60764e0e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:04.867882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568f834-4822-4938-b4bd-becba9e09d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:04.869062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568f834-4822-4938-b4bd-becba9e09d15 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:04.869551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568f834-4822-4938-b4bd-becba9e09d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.870133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568f834-4822-4938-b4bd-becba9e09d15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.882790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568f834-4822-4938-b4bd-becba9e09d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:04.891509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-c568f834-4822-4938-b4bd-becba9e09d15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:04.893022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:04.893954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:04.894807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:04.894807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:04.894807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:04.894807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:04.894807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:04.894807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:04.894807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:04.894807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:04.894807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:04.894807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:04.894807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:04.894807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:04.897361 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 303/614] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:04.904945 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568f834-4822-4938-b4bd-becba9e09d15 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:04.947279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13574460-49ee-4cc2-83f9-a977fca4427b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:04.972257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:04.986589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3475bb0-c9c7-4177-985c-926c07a97411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:04.987842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3475bb0-c9c7-4177-985c-926c07a97411 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:04.987842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3475bb0-c9c7-4177-985c-926c07a97411 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.988037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3475bb0-c9c7-4177-985c-926c07a97411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.999884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3475bb0-c9c7-4177-985c-926c07a97411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:05.001398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3475bb0-c9c7-4177-985c-926c07a97411 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-d3475bb0-c9c7-4177-985c-926c07a97411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:05.018806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:05.019724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:05.020643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:05.020643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:05.020643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:05.020643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:05.020643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 312/615] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:05.025447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3475bb0-c9c7-4177-985c-926c07a97411 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:05.026658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568f834-4822-4938-b4bd-becba9e09d15 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:05.062763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1a3c1-a296-4aea-a34f-e4f8e1729eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:05.063675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1a3c1-a296-4aea-a34f-e4f8e1729eab None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:05.064101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1a3c1-a296-4aea-a34f-e4f8e1729eab None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:05.064540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1a3c1-a296-4aea-a34f-e4f8e1729eab None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:05.065490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1a3c1-a296-4aea-a34f-e4f8e1729eab None admin] RBAC: Adding query filter params (name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:05.065957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1a3c1-a296-4aea-a34f-e4f8e1729eab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:05.067557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1a3c1-a296-4aea-a34f-e4f8e1729eab None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:05.076842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 304/616] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:27:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:05.082524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1a3c1-a296-4aea-a34f-e4f8e1729eab None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:05.129948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3475bb0-c9c7-4177-985c-926c07a97411 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:05.131821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:05.197225 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1a3c1-a296-4aea-a34f-e4f8e1729eab None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:05.216877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:05.223299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca37e8-42bc-43bb-a0b8-be246eb28a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:05.223938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca37e8-42bc-43bb-a0b8-be246eb28a54 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:05.224401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca37e8-42bc-43bb-a0b8-be246eb28a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:05.224691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca37e8-42bc-43bb-a0b8-be246eb28a54 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:05.225438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca37e8-42bc-43bb-a0b8-be246eb28a54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:05.225792 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca37e8-42bc-43bb-a0b8-be246eb28a54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:05.227361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca37e8-42bc-43bb-a0b8-be246eb28a54 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:05.238408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc6a4a-7ec6-4b98-8026-9f56ba06b0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:05.239572 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc6a4a-7ec6-4b98-8026-9f56ba06b0af None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:05.239997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc6a4a-7ec6-4b98-8026-9f56ba06b0af None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:05.240432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc6a4a-7ec6-4b98-8026-9f56ba06b0af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:05.252359 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc6a4a-7ec6-4b98-8026-9f56ba06b0af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:05.254032 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc6a4a-7ec6-4b98-8026-9f56ba06b0af None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:05.261039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 313/617] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:05.266441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca37e8-42bc-43bb-a0b8-be246eb28a54 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-64cc6a4a-7ec6-4b98-8026-9f56ba06b0af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:05.273805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:05.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:05.276536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:05.276536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:05.276536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:05.276536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:05.278591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 305/618] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:05.284308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc6a4a-7ec6-4b98-8026-9f56ba06b0af None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:05.409254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc6a4a-7ec6-4b98-8026-9f56ba06b0af None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:05.426099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca37e8-42bc-43bb-a0b8-be246eb28a54 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:05.448473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f2772-26e9-42b6-aed0-cfe1b2079ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:05.449568 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f2772-26e9-42b6-aed0-cfe1b2079ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:05.450736 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f2772-26e9-42b6-aed0-cfe1b2079ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:05.451485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f2772-26e9-42b6-aed0-cfe1b2079ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:05.453769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f2772-26e9-42b6-aed0-cfe1b2079ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:05.454484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f2772-26e9-42b6-aed0-cfe1b2079ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:05.456891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f2772-26e9-42b6-aed0-cfe1b2079ee6 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:05.479092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 306/619] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:05.485091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f2772-26e9-42b6-aed0-cfe1b2079ee6 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:05.498967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43fc98-274b-41ee-a968-272843ecdb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:05.500053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43fc98-274b-41ee-a968-272843ecdb12 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:05.500539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43fc98-274b-41ee-a968-272843ecdb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:05.501035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43fc98-274b-41ee-a968-272843ecdb12 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:05.515998 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43fc98-274b-41ee-a968-272843ecdb12 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:05.518994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43fc98-274b-41ee-a968-272843ecdb12 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-8f43fc98-274b-41ee-a968-272843ecdb12 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:05.541819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:05.542922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:05.543851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:05.543851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:05.543851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:05.543851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:05.543851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:05.543851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:05.543851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:05.543851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:05.543851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:05.543851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:05.547013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 314/620] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Jul 25 12:27:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:05.554633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:05.556329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43fc98-274b-41ee-a968-272843ecdb12 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:05.607774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:05.684346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f2772-26e9-42b6-aed0-cfe1b2079ee6 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:05.745892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715283d-8fa8-46c4-9780-d0adb89d0c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:05.746705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715283d-8fa8-46c4-9780-d0adb89d0c11 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:05.746987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715283d-8fa8-46c4-9780-d0adb89d0c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:05.747264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715283d-8fa8-46c4-9780-d0adb89d0c11 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:05.749613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43fc98-274b-41ee-a968-272843ecdb12 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:05.765740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715283d-8fa8-46c4-9780-d0adb89d0c11 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:05.767220 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715283d-8fa8-46c4-9780-d0adb89d0c11 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-9715283d-8fa8-46c4-9780-d0adb89d0c11 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:05.790739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:05.791743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:05.792667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:05.792667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:05.792667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:05.792667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:05.792667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:05.792667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:05.792667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:05.792667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:05.792667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:27:05.792667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:05.792667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 307/621] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:05.797526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715283d-8fa8-46c4-9780-d0adb89d0c11 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:05.815256 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e5856-fe4b-4c85-b5cd-487b572d4b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:05.815952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e5856-fe4b-4c85-b5cd-487b572d4b3b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:05.816237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e5856-fe4b-4c85-b5cd-487b572d4b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:05.816515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e5856-fe4b-4c85-b5cd-487b572d4b3b None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:05.817788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e5856-fe4b-4c85-b5cd-487b572d4b3b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:05.818363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e5856-fe4b-4c85-b5cd-487b572d4b3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:05.819851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e5856-fe4b-4c85-b5cd-487b572d4b3b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:05.854371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 315/622] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Jul 25 12:27:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:05.859192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e5856-fe4b-4c85-b5cd-487b572d4b3b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:05.952323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:05.973101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715283d-8fa8-46c4-9780-d0adb89d0c11 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:06.000721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:06.031362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e5856-fe4b-4c85-b5cd-487b572d4b3b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:06.034919 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8e676-83f5-4d0f-a81d-b35e0df63b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:06.035594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8e676-83f5-4d0f-a81d-b35e0df63b5c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:06.035925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8e676-83f5-4d0f-a81d-b35e0df63b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:06.036208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8e676-83f5-4d0f-a81d-b35e0df63b5c None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:06.037051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8e676-83f5-4d0f-a81d-b35e0df63b5c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:06.037358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8e676-83f5-4d0f-a81d-b35e0df63b5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:06.038806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8e676-83f5-4d0f-a81d-b35e0df63b5c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:06.068511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 308/623] 10.222.0.60 () {56 vars in 1135 bytes} [Mon Jul 25 12:27:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:06.073885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8e676-83f5-4d0f-a81d-b35e0df63b5c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:06.091701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c1bfe-3a8b-4c3a-a65b-c581d912c253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:06.092380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c1bfe-3a8b-4c3a-a65b-c581d912c253 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:06.092672 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c1bfe-3a8b-4c3a-a65b-c581d912c253 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:06.092954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c1bfe-3a8b-4c3a-a65b-c581d912c253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:06.193961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8e676-83f5-4d0f-a81d-b35e0df63b5c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:06.221413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c1bfe-3a8b-4c3a-a65b-c581d912c253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:06.222828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c1bfe-3a8b-4c3a-a65b-c581d912c253 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:06.251949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c8ea9-fea7-4212-8268-a5893d1bc000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:06.252751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c8ea9-fea7-4212-8268-a5893d1bc000 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:06.253035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c8ea9-fea7-4212-8268-a5893d1bc000 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:06.253308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c8ea9-fea7-4212-8268-a5893d1bc000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:06.271347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c8ea9-fea7-4212-8268-a5893d1bc000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:06.272814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c8ea9-fea7-4212-8268-a5893d1bc000 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:06.281839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-380c8ea9-fea7-4212-8268-a5893d1bc000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:06.294511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:06.295441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:06.296349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:06.296349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:06.296349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:06.296349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:06.296349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 309/624] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:06.297789 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158c273-4bff-4000-9d10-4d8adabd46ec None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:06.297978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158c273-4bff-4000-9d10-4d8adabd46ec None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:06.302715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158c273-4bff-4000-9d10-4d8adabd46ec None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:06.304735 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 310/625] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:06.309360 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158c273-4bff-4000-9d10-4d8adabd46ec None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:06.358292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:06.376287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 316/626] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:27:06.377884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d33c6e-2dd7-4d30-bbc2-6962d67df230 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:06.378442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d33c6e-2dd7-4d30-bbc2-6962d67df230 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:06.378921 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d33c6e-2dd7-4d30-bbc2-6962d67df230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:06.398542 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-16d33c6e-2dd7-4d30-bbc2-6962d67df230 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:06.400182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-16d33c6e-2dd7-4d30-bbc2-6962d67df230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:06.422017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158c273-4bff-4000-9d10-4d8adabd46ec None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:06.461722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0214076-7d36-45d3-94e6-422f6ce9f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:06.462381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0214076-7d36-45d3-94e6-422f6ce9f66d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:06.462709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0214076-7d36-45d3-94e6-422f6ce9f66d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:06.462983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0214076-7d36-45d3-94e6-422f6ce9f66d None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:06.463788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0214076-7d36-45d3-94e6-422f6ce9f66d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:06.464094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0214076-7d36-45d3-94e6-422f6ce9f66d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:06.465509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0214076-7d36-45d3-94e6-422f6ce9f66d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:06.482271 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 317/628] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:06.482427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 311/628] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:06.483630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0214076-7d36-45d3-94e6-422f6ce9f66d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:06.484001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d33c6e-2dd7-4d30-bbc2-6962d67df230 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:06.545849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:06.592545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0214076-7d36-45d3-94e6-422f6ce9f66d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:06.623560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d33c6e-2dd7-4d30-bbc2-6962d67df230 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:06.628717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8761c-9e00-4680-81d5-3f4c1c4c5c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:06.629583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8761c-9e00-4680-81d5-3f4c1c4c5c32 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:06.629952 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8761c-9e00-4680-81d5-3f4c1c4c5c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:06.630310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8761c-9e00-4680-81d5-3f4c1c4c5c32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:06.638510 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:06.643243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8761c-9e00-4680-81d5-3f4c1c4c5c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:06.644850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8761c-9e00-4680-81d5-3f4c1c4c5c32 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-a1f8761c-9e00-4680-81d5-3f4c1c4c5c32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:06.663630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:06.664651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:06.665543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:06.665543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:06.665543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:06.665543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:06.670209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 312/629] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:06.671168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8761c-9e00-4680-81d5-3f4c1c4c5c32 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:06.672004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a5be6-8570-405d-a13a-bf160f7cf49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:06.673005 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a5be6-8570-405d-a13a-bf160f7cf49d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:06.673467 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a5be6-8570-405d-a13a-bf160f7cf49d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:06.673944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a5be6-8570-405d-a13a-bf160f7cf49d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:06.686596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a5be6-8570-405d-a13a-bf160f7cf49d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:06.688857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a5be6-8570-405d-a13a-bf160f7cf49d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-e32a5be6-8570-405d-a13a-bf160f7cf49d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:06.708095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:06.709065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:06.710003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:06.710003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:06.710003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:06.710003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:06.710003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:06.710003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:06.710003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:06.710003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:06.710003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 318/630] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:06.713966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a5be6-8570-405d-a13a-bf160f7cf49d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:06.787715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8761c-9e00-4680-81d5-3f4c1c4c5c32 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:06.800867 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:06.825823 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ab899-7f78-45af-8068-7f4d9162c052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:06.826753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ab899-7f78-45af-8068-7f4d9162c052 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:06.827033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ab899-7f78-45af-8068-7f4d9162c052 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:06.827300 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ab899-7f78-45af-8068-7f4d9162c052 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:06.838674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a5be6-8570-405d-a13a-bf160f7cf49d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:06.840413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ab899-7f78-45af-8068-7f4d9162c052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-df9ab899-7f78-45af-8068-7f4d9162c052 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:06.850524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:06.851462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 313/631] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:06.858231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ab899-7f78-45af-8068-7f4d9162c052 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:06.874969 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b28ac-0fda-4249-ba17-fc038b1fb897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:06.875638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b28ac-0fda-4249-ba17-fc038b1fb897 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:06.875924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b28ac-0fda-4249-ba17-fc038b1fb897 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:06.876205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b28ac-0fda-4249-ba17-fc038b1fb897 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:06.876942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b28ac-0fda-4249-ba17-fc038b1fb897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:06.877252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b28ac-0fda-4249-ba17-fc038b1fb897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:06.878664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b28ac-0fda-4249-ba17-fc038b1fb897 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:06.890820 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 319/632] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:06.892083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f0aee-3126-49a9-ac31-cccc592d24d1 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:06.892369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f0aee-3126-49a9-ac31-cccc592d24d1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:06.892653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f0aee-3126-49a9-ac31-cccc592d24d1 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:06.894087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 320/633] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:06.898772 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f0aee-3126-49a9-ac31-cccc592d24d1 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:06.930975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:06.975318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ab899-7f78-45af-8068-7f4d9162c052 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:07.009810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f0aee-3126-49a9-ac31-cccc592d24d1 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:07.011574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30778dac-15fa-42de-a734-0868f733c884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:07.012429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30778dac-15fa-42de-a734-0868f733c884 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.012909 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30778dac-15fa-42de-a734-0868f733c884 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.013242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30778dac-15fa-42de-a734-0868f733c884 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.014076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30778dac-15fa-42de-a734-0868f733c884 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:07.014386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30778dac-15fa-42de-a734-0868f733c884 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:07.015835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30778dac-15fa-42de-a734-0868f733c884 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:07.028627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 314/634] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:07.029423 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30778dac-15fa-42de-a734-0868f733c884 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:07.047912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30681b8-3bdf-4180-8a68-8a84ade8e1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:07.048509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30681b8-3bdf-4180-8a68-8a84ade8e1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.048792 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30681b8-3bdf-4180-8a68-8a84ade8e1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.049069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30681b8-3bdf-4180-8a68-8a84ade8e1e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.049823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30681b8-3bdf-4180-8a68-8a84ade8e1e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:07.050159 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30681b8-3bdf-4180-8a68-8a84ade8e1e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:07.051518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30681b8-3bdf-4180-8a68-8a84ade8e1e7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:07.058416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:07.059464 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 321/635] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:27:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:07.060587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cde287-d4e8-494b-9b6c-f97d0135ec64 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.060877 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cde287-d4e8-494b-9b6c-f97d0135ec64 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.061164 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cde287-d4e8-494b-9b6c-f97d0135ec64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.078309 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-50cde287-d4e8-494b-9b6c-f97d0135ec64 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:07.079494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-50cde287-d4e8-494b-9b6c-f97d0135ec64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:07.128286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30778dac-15fa-42de-a734-0868f733c884 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:07.144193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 322/636] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:07.145455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427065c-d076-48ff-8424-4e296ac1f287 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.145735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427065c-d076-48ff-8424-4e296ac1f287 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.146033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427065c-d076-48ff-8424-4e296ac1f287 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.147270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 323/637] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:07.147938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427065c-d076-48ff-8424-4e296ac1f287 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:07.170330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e232b-1b2a-42d0-8d78-1254cc17d8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:07.171127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e232b-1b2a-42d0-8d78-1254cc17d8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.171403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e232b-1b2a-42d0-8d78-1254cc17d8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.171678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e232b-1b2a-42d0-8d78-1254cc17d8e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.174971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:07.179057 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e232b-1b2a-42d0-8d78-1254cc17d8e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:07.180473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e232b-1b2a-42d0-8d78-1254cc17d8e1 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-d93e232b-1b2a-42d0-8d78-1254cc17d8e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:07.196496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:07.197494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:07.203305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:07.203305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:07.203305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:07.203305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:07.203305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:07.203305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:07.203305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:07.203305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:07.203305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 315/638] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:07.203305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e232b-1b2a-42d0-8d78-1254cc17d8e1 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:07.254695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427065c-d076-48ff-8424-4e296ac1f287 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:07.288149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515805d5-c9fb-43d8-9456-7ca4f9f10a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:07.289175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515805d5-c9fb-43d8-9456-7ca4f9f10a6e None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.289557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515805d5-c9fb-43d8-9456-7ca4f9f10a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.289731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515805d5-c9fb-43d8-9456-7ca4f9f10a6e None admin] PATH_INFO: `/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/e56b0d48e08c4f778a7c509c10edc871/roles/d9c5c1adb796486d83edcd6e728a5de9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.289835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:07.302565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e232b-1b2a-42d0-8d78-1254cc17d8e1 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:07.305461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515805d5-c9fb-43d8-9456-7ca4f9f10a6e None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff5a41f8ca6438295c512792d2022e4, user_id=e56b0d48e08c4f778a7c509c10edc871, role_id=d9c5c1adb796486d83edcd6e728a5de9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:07.306926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515805d5-c9fb-43d8-9456-7ca4f9f10a6e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:07.318937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 324/639] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:27:07 2022] PUT /identity/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/e56b0d48e08c4f778a7c509c10edc871/roles/d9c5c1adb796486d83edcd6e728a5de9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:27:07.320190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd2cb8-a346-4599-8c12-c8717c109309 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.320470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd2cb8-a346-4599-8c12-c8717c109309 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.320742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd2cb8-a346-4599-8c12-c8717c109309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.340258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-95cd2cb8-a346-4599-8c12-c8717c109309 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:07.341536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-95cd2cb8-a346-4599-8c12-c8717c109309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:07.350432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927e52b-ec0f-41db-981e-cfb60f2ae3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:07.351350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e52b-ec0f-41db-981e-cfb60f2ae3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.351700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e52b-ec0f-41db-981e-cfb60f2ae3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.352043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e52b-ec0f-41db-981e-cfb60f2ae3d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.364744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927e52b-ec0f-41db-981e-cfb60f2ae3d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:07.366397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927e52b-ec0f-41db-981e-cfb60f2ae3d8 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-e927e52b-ec0f-41db-981e-cfb60f2ae3d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:07.385668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:07.386878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:07.387962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:07.387962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:07.387962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:07.387962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:07.393741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 316/640] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:07.393741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927e52b-ec0f-41db-981e-cfb60f2ae3d8 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:07.425951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 325/641] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:07.425951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd2cb8-a346-4599-8c12-c8717c109309 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:07.426799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:07.510654 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927e52b-ec0f-41db-981e-cfb60f2ae3d8 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:07.540501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd2cb8-a346-4599-8c12-c8717c109309 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:07.543767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdba719-7aed-4183-aa45-e420f92ca5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:07.544570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdba719-7aed-4183-aa45-e420f92ca5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.544856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdba719-7aed-4183-aa45-e420f92ca5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.545135 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdba719-7aed-4183-aa45-e420f92ca5e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.555454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdba719-7aed-4183-aa45-e420f92ca5e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-6fdba719-7aed-4183-aa45-e420f92ca5e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:07.563123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:07.564127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:07.564978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 317/642] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:07.565688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdba719-7aed-4183-aa45-e420f92ca5e9 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:07.569397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:07.584771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cc0a7-b686-40e7-8bfd-b57d19e2dea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:07.585650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cc0a7-b686-40e7-8bfd-b57d19e2dea8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.586137 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cc0a7-b686-40e7-8bfd-b57d19e2dea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.586418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cc0a7-b686-40e7-8bfd-b57d19e2dea8 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.587214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cc0a7-b686-40e7-8bfd-b57d19e2dea8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:07.587523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cc0a7-b686-40e7-8bfd-b57d19e2dea8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:07.589515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cc0a7-b686-40e7-8bfd-b57d19e2dea8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:07.601813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 326/643] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:07.607643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cc0a7-b686-40e7-8bfd-b57d19e2dea8 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:07.675235 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:07.679807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdba719-7aed-4183-aa45-e420f92ca5e9 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:07.716059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc83480-4e34-46f7-8b31-3f58e0087c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:07.716721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc83480-4e34-46f7-8b31-3f58e0087c9e None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.717009 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc83480-4e34-46f7-8b31-3f58e0087c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.717286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc83480-4e34-46f7-8b31-3f58e0087c9e None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.718039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc83480-4e34-46f7-8b31-3f58e0087c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:07.718351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc83480-4e34-46f7-8b31-3f58e0087c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:07.719735 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc83480-4e34-46f7-8b31-3f58e0087c9e None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:07.724092 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cc0a7-b686-40e7-8bfd-b57d19e2dea8 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:07.736347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 318/644] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:07.736347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc83480-4e34-46f7-8b31-3f58e0087c9e None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:07.769132 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507106d-2093-4580-adad-4f1c59ebe43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:07.769958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507106d-2093-4580-adad-4f1c59ebe43e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.770331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507106d-2093-4580-adad-4f1c59ebe43e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.770616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507106d-2093-4580-adad-4f1c59ebe43e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.781290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507106d-2093-4580-adad-4f1c59ebe43e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-3507106d-2093-4580-adad-4f1c59ebe43e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:07.788953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:07.789875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:07.790824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 327/645] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:07.792392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507106d-2093-4580-adad-4f1c59ebe43e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:07.802112 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:07.841444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc83480-4e34-46f7-8b31-3f58e0087c9e None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:07.875882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302dd6f-4880-40e6-8c1b-fec8b0b04263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:07.876545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302dd6f-4880-40e6-8c1b-fec8b0b04263 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.876824 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302dd6f-4880-40e6-8c1b-fec8b0b04263 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.877093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302dd6f-4880-40e6-8c1b-fec8b0b04263 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.877830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302dd6f-4880-40e6-8c1b-fec8b0b04263 None admin] RBAC: Adding query filter params (name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:07.878163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302dd6f-4880-40e6-8c1b-fec8b0b04263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:07.879511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302dd6f-4880-40e6-8c1b-fec8b0b04263 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:07.887616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 319/646] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:27:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:07.888387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302dd6f-4880-40e6-8c1b-fec8b0b04263 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:07.910127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507106d-2093-4580-adad-4f1c59ebe43e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:07.935127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:07.947859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6b55a-4b52-4a33-a0ac-195bfab89a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:07.948519 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6b55a-4b52-4a33-a0ac-195bfab89a0d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:07.948806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6b55a-4b52-4a33-a0ac-195bfab89a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:07.949325 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6b55a-4b52-4a33-a0ac-195bfab89a0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:07.949939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6b55a-4b52-4a33-a0ac-195bfab89a0d None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=ccf2dedaf7a449d2b51b9e9f3b420091) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:07.950274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6b55a-4b52-4a33-a0ac-195bfab89a0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:07.951876 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6b55a-4b52-4a33-a0ac-195bfab89a0d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:07.965475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 328/647] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:07 2022] GET /identity/v3/role_assignments?user.id=ccf2dedaf7a449d2b51b9e9f3b420091&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:07.965475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6b55a-4b52-4a33-a0ac-195bfab89a0d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:08.024805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302dd6f-4880-40e6-8c1b-fec8b0b04263 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:08.061959 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:08.078644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044ac4f-1b6d-40ca-8b1b-21bab84bebe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:08.079723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044ac4f-1b6d-40ca-8b1b-21bab84bebe5 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:08.080011 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044ac4f-1b6d-40ca-8b1b-21bab84bebe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:08.080289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044ac4f-1b6d-40ca-8b1b-21bab84bebe5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:08.086642 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6b55a-4b52-4a33-a0ac-195bfab89a0d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:08.092606 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044ac4f-1b6d-40ca-8b1b-21bab84bebe5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:08.094045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044ac4f-1b6d-40ca-8b1b-21bab84bebe5 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-a044ac4f-1b6d-40ca-8b1b-21bab84bebe5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:08.112066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:08.113245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:08.115318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:08.115318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:08.115318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:08.115318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:08.115318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:08.115318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:08.115318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:08.115318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:08.115318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:27:08.115318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:08.116554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 320/648] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:08.116554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044ac4f-1b6d-40ca-8b1b-21bab84bebe5 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:08.136664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1f9b6-9511-4c56-bda0-458277537c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:08.137666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1f9b6-9511-4c56-bda0-458277537c39 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:08.139688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1f9b6-9511-4c56-bda0-458277537c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:08.139688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1f9b6-9511-4c56-bda0-458277537c39 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:08.139688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1f9b6-9511-4c56-bda0-458277537c39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:08.139688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1f9b6-9511-4c56-bda0-458277537c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:08.141159 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1f9b6-9511-4c56-bda0-458277537c39 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:08.153283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 329/649] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:27:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:08.154220 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1f9b6-9511-4c56-bda0-458277537c39 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:08.217668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:08.241378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044ac4f-1b6d-40ca-8b1b-21bab84bebe5 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:08.283538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88741baf-2293-4548-b287-5bab667ed6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:08.284333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88741baf-2293-4548-b287-5bab667ed6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:08.284614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88741baf-2293-4548-b287-5bab667ed6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:08.284882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88741baf-2293-4548-b287-5bab667ed6e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:08.297643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1f9b6-9511-4c56-bda0-458277537c39 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:08.299644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88741baf-2293-4548-b287-5bab667ed6e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:08.301923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88741baf-2293-4548-b287-5bab667ed6e9 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-88741baf-2293-4548-b287-5bab667ed6e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:08.322171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:08.323663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:08.324999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:08.324999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:08.324999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:08.324999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:08.324999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 321/650] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:08.325284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88741baf-2293-4548-b287-5bab667ed6e9 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:08.336628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb30283-4320-4771-91d3-6753db4c476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:08.337696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb30283-4320-4771-91d3-6753db4c476b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:08.338009 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb30283-4320-4771-91d3-6753db4c476b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:08.338844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb30283-4320-4771-91d3-6753db4c476b None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:08.340289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb30283-4320-4771-91d3-6753db4c476b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:08.340603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb30283-4320-4771-91d3-6753db4c476b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:08.342275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb30283-4320-4771-91d3-6753db4c476b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:08.362838 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 330/651] 10.222.0.60 () {56 vars in 1137 bytes} [Mon Jul 25 12:27:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:08.363100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:08.363980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb30283-4320-4771-91d3-6753db4c476b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:08.436747 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88741baf-2293-4548-b287-5bab667ed6e9 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:08.470941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb30283-4320-4771-91d3-6753db4c476b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:08.474180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7edb8-2d5c-42a7-a98b-890a6a83cd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:08.474983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7edb8-2d5c-42a7-a98b-890a6a83cd5d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:08.475260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7edb8-2d5c-42a7-a98b-890a6a83cd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:08.475532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7edb8-2d5c-42a7-a98b-890a6a83cd5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:08.482056 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:08.491094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7edb8-2d5c-42a7-a98b-890a6a83cd5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:08.493097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7edb8-2d5c-42a7-a98b-890a6a83cd5d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:08.515372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6d49f-2b7b-410b-a55a-7f72bbdc7b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-53d7edb8-2d5c-42a7-a98b-890a6a83cd5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:08.515948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:08.517590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:08.518594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:08.518594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:08.518594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:08.518594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:08.518812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6d49f-2b7b-410b-a55a-7f72bbdc7b8c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:08.518812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6d49f-2b7b-410b-a55a-7f72bbdc7b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:08.518812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6d49f-2b7b-410b-a55a-7f72bbdc7b8c None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:08.519008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 322/652] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:08.519008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7edb8-2d5c-42a7-a98b-890a6a83cd5d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:08.524005 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6d49f-2b7b-410b-a55a-7f72bbdc7b8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:08.524330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6d49f-2b7b-410b-a55a-7f72bbdc7b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:08.526453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6d49f-2b7b-410b-a55a-7f72bbdc7b8c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:08.542108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 331/653] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:08.542945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6d49f-2b7b-410b-a55a-7f72bbdc7b8c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:08.630371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:08.645946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7edb8-2d5c-42a7-a98b-890a6a83cd5d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:08.672412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6d49f-2b7b-410b-a55a-7f72bbdc7b8c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:08.688881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6d6b7-4d5b-4bb7-8d7f-13f9f0fa27d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:08.689768 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6d6b7-4d5b-4bb7-8d7f-13f9f0fa27d4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:08.690105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6d6b7-4d5b-4bb7-8d7f-13f9f0fa27d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:08.690492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6d6b7-4d5b-4bb7-8d7f-13f9f0fa27d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:08.714807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0017f0-2ba3-46bc-ad6a-a93adb1c290d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:08.715695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0017f0-2ba3-46bc-ad6a-a93adb1c290d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:08.716410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0017f0-2ba3-46bc-ad6a-a93adb1c290d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:08.716844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0017f0-2ba3-46bc-ad6a-a93adb1c290d None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:08.718019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0017f0-2ba3-46bc-ad6a-a93adb1c290d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:08.718890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0017f0-2ba3-46bc-ad6a-a93adb1c290d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:08.720947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0017f0-2ba3-46bc-ad6a-a93adb1c290d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:08.737797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 332/654] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:08.738960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0017f0-2ba3-46bc-ad6a-a93adb1c290d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:08.770129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6d6b7-4d5b-4bb7-8d7f-13f9f0fa27d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:08.771764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6d6b7-4d5b-4bb7-8d7f-13f9f0fa27d4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:08.776007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:08.874327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:08.874662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0017f0-2ba3-46bc-ad6a-a93adb1c290d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:08.895090 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 323/655] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:27:08.896029 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6d6b7-4d5b-4bb7-8d7f-13f9f0fa27d4 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:08.921504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2876e-2260-47eb-ab07-f512ec7c7f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:08.922988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2876e-2260-47eb-ab07-f512ec7c7f96 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:08.923967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2876e-2260-47eb-ab07-f512ec7c7f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:08.924727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2876e-2260-47eb-ab07-f512ec7c7f96 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:08.936447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2876e-2260-47eb-ab07-f512ec7c7f96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:08.938186 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2876e-2260-47eb-ab07-f512ec7c7f96 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-63c2876e-2260-47eb-ab07-f512ec7c7f96 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:08.958527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:08.959498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:08.960442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:08.960442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:08.960442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:08.960442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:08.960442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:08.960442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:08.960442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:08.960442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:08.960442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 12:27:08.960442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:08.960442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 333/656] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:08.961047 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2876e-2260-47eb-ab07-f512ec7c7f96 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:09.027350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6d6b7-4d5b-4bb7-8d7f-13f9f0fa27d4 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:09.055975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:09.073457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2876e-2260-47eb-ab07-f512ec7c7f96 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:09.077625 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb1a6f-a599-4535-8426-806d03dbfe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:09.078642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb1a6f-a599-4535-8426-806d03dbfe71 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:09.079105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb1a6f-a599-4535-8426-806d03dbfe71 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:09.079531 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb1a6f-a599-4535-8426-806d03dbfe71 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:09.089773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb1a6f-a599-4535-8426-806d03dbfe71 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:09.091951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb1a6f-a599-4535-8426-806d03dbfe71 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:09.110845 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cec27-550b-4ff1-b8f0-bb0d3c3197fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:09.111839 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cec27-550b-4ff1-b8f0-bb0d3c3197fd None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:09.112218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cec27-550b-4ff1-b8f0-bb0d3c3197fd None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:09.112590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cec27-550b-4ff1-b8f0-bb0d3c3197fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-d1cb1a6f-a599-4535-8426-806d03dbfe71 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:09.113091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:09.114211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:09.115380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:09.115380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:09.115380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:09.115380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:09.115380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:09.115380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:09.115380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:09.115380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:09.115380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 12:27:09.115380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:09.118355 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 324/657] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:09.119388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb1a6f-a599-4535-8426-806d03dbfe71 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:09.132032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cec27-550b-4ff1-b8f0-bb0d3c3197fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:09.133490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cec27-550b-4ff1-b8f0-bb0d3c3197fd None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-256cec27-550b-4ff1-b8f0-bb0d3c3197fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:09.159295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:09.160374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:09.161455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:09.161455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:09.161455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:09.161455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:09.164297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 334/658] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:09.165267 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cec27-550b-4ff1-b8f0-bb0d3c3197fd None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:09.188598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:09.246808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb1a6f-a599-4535-8426-806d03dbfe71 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:09.285353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cec27-550b-4ff1-b8f0-bb0d3c3197fd None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:09.286404 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e553abc-e9a9-4067-b0ab-d6497213da26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:09.287193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e553abc-e9a9-4067-b0ab-d6497213da26 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:09.287569 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e553abc-e9a9-4067-b0ab-d6497213da26 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:09.287944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e553abc-e9a9-4067-b0ab-d6497213da26 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:09.289407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e553abc-e9a9-4067-b0ab-d6497213da26 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:09.289407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e553abc-e9a9-4067-b0ab-d6497213da26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:09.290850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e553abc-e9a9-4067-b0ab-d6497213da26 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:09.310031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 325/659] 10.222.0.60 () {56 vars in 1137 bytes} [Mon Jul 25 12:27:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:09.311073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e553abc-e9a9-4067-b0ab-d6497213da26 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:09.328604 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf7eb6-f9d7-4895-8b64-5139c72f3a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:09.329508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf7eb6-f9d7-4895-8b64-5139c72f3a40 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:09.329968 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf7eb6-f9d7-4895-8b64-5139c72f3a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:09.330397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf7eb6-f9d7-4895-8b64-5139c72f3a40 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:09.331473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf7eb6-f9d7-4895-8b64-5139c72f3a40 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:09.331955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf7eb6-f9d7-4895-8b64-5139c72f3a40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:09.333527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf7eb6-f9d7-4895-8b64-5139c72f3a40 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:09.342455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:09.353130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 335/660] 10.222.0.60 () {56 vars in 1139 bytes} [Mon Jul 25 12:27:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:09.354904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf7eb6-f9d7-4895-8b64-5139c72f3a40 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:09.426737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e553abc-e9a9-4067-b0ab-d6497213da26 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:09.434692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:09.456388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf7eb6-f9d7-4895-8b64-5139c72f3a40 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:09.464335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6022ef-9f41-4050-a50a-2b830f30ab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:09.464992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6022ef-9f41-4050-a50a-2b830f30ab33 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:09.465272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6022ef-9f41-4050-a50a-2b830f30ab33 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:09.465559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6022ef-9f41-4050-a50a-2b830f30ab33 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:09.466404 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6022ef-9f41-4050-a50a-2b830f30ab33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:09.466704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6022ef-9f41-4050-a50a-2b830f30ab33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:09.468172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6022ef-9f41-4050-a50a-2b830f30ab33 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:09.485251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 326/661] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:09.487046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166aa5d-9d8d-4727-a1db-2d640d68f041 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:09.487333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166aa5d-9d8d-4727-a1db-2d640d68f041 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:09.487608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166aa5d-9d8d-4727-a1db-2d640d68f041 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:09.489959 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 327/662] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:09.489959 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166aa5d-9d8d-4727-a1db-2d640d68f041 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:09.495312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b27c6-13fa-40ce-950d-fe5d26398117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:09.495960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b27c6-13fa-40ce-950d-fe5d26398117 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:09.496244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b27c6-13fa-40ce-950d-fe5d26398117 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:09.496518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b27c6-13fa-40ce-950d-fe5d26398117 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:09.562987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:09.570232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b27c6-13fa-40ce-950d-fe5d26398117 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:09.571693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b27c6-13fa-40ce-950d-fe5d26398117 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:09.599418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166aa5d-9d8d-4727-a1db-2d640d68f041 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:09.636960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eba05-1f88-474f-8f1d-ab6e6b72da30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:09.637603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eba05-1f88-474f-8f1d-ab6e6b72da30 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:09.637888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eba05-1f88-474f-8f1d-ab6e6b72da30 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:09.638215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eba05-1f88-474f-8f1d-ab6e6b72da30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:09.673574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:09.681782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 336/663] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:27:09.681782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b27c6-13fa-40ce-950d-fe5d26398117 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:09.716412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345eba05-1f88-474f-8f1d-ab6e6b72da30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:09.717863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345eba05-1f88-474f-8f1d-ab6e6b72da30 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:09.781585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:09.788648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b27c6-13fa-40ce-950d-fe5d26398117 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:09.822174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 328/664] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:27:09.823746 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f558ab-34d5-457b-8083-b2d921944bd5 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:09.824034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f558ab-34d5-457b-8083-b2d921944bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:09.824307 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f558ab-34d5-457b-8083-b2d921944bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:09.828665 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4eefa-7724-4699-b17d-df4592abd06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:09.829477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4eefa-7724-4699-b17d-df4592abd06d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:09.829760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4eefa-7724-4699-b17d-df4592abd06d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:09.830068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4eefa-7724-4699-b17d-df4592abd06d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:09.837641 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4eefa-7724-4699-b17d-df4592abd06d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:09.839167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4eefa-7724-4699-b17d-df4592abd06d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:09.842570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-d8f558ab-34d5-457b-8083-b2d921944bd5 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:09.843755 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d8f558ab-34d5-457b-8083-b2d921944bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-97e4eefa-7724-4699-b17d-df4592abd06d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:09.853180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:09.854793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:09.855679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:09.855679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:09.855679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:09.855679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:09.855679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:09.855679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:09.855679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:09.855679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:09.855679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 337/665] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:09.855679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4eefa-7724-4699-b17d-df4592abd06d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:09.911451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:09.921121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 329/666] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:09.922222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f558ab-34d5-457b-8083-b2d921944bd5 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:09.971052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4eefa-7724-4699-b17d-df4592abd06d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:10.006562 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f725284-309d-4a4a-bfd8-8366c208866e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:10.007466 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f725284-309d-4a4a-bfd8-8366c208866e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.007745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f725284-309d-4a4a-bfd8-8366c208866e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.008017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f725284-309d-4a4a-bfd8-8366c208866e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.021473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f725284-309d-4a4a-bfd8-8366c208866e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:10.022952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f725284-309d-4a4a-bfd8-8366c208866e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:10.032287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f558ab-34d5-457b-8083-b2d921944bd5 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-5f725284-309d-4a4a-bfd8-8366c208866e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:10.041016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:10.042660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:10.048063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:10.048063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:10.048063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:10.048063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:10.048063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 338/667] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:10.048063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f725284-309d-4a4a-bfd8-8366c208866e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:10.057559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:10.075531 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf3a22-eab8-4f69-a793-a66f559f13f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:10.076384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf3a22-eab8-4f69-a793-a66f559f13f4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.076665 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf3a22-eab8-4f69-a793-a66f559f13f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.076938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf3a22-eab8-4f69-a793-a66f559f13f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.089647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf3a22-eab8-4f69-a793-a66f559f13f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:10.092417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf3a22-eab8-4f69-a793-a66f559f13f4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-fbaf3a22-eab8-4f69-a793-a66f559f13f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:10.112908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:10.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:10.114723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:10.114723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:10.114723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:10.114723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:10.114723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 330/668] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:10.115003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf3a22-eab8-4f69-a793-a66f559f13f4 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:10.154809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f725284-309d-4a4a-bfd8-8366c208866e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:10.165120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:10.192258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1f944-b4a9-497a-9d84-7627d34017d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:10.192927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1f944-b4a9-497a-9d84-7627d34017d0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.193205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1f944-b4a9-497a-9d84-7627d34017d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.193478 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1f944-b4a9-497a-9d84-7627d34017d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.194294 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1f944-b4a9-497a-9d84-7627d34017d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:10.194601 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1f944-b4a9-497a-9d84-7627d34017d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:10.196198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1f944-b4a9-497a-9d84-7627d34017d0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:10.207996 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 339/669] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:10.208771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1f944-b4a9-497a-9d84-7627d34017d0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:10.219717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf3a22-eab8-4f69-a793-a66f559f13f4 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:10.253212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4285cd-2725-4d54-8c95-b017030040ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:10.253964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4285cd-2725-4d54-8c95-b017030040ee None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.254258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4285cd-2725-4d54-8c95-b017030040ee None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.254525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4285cd-2725-4d54-8c95-b017030040ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.264304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4285cd-2725-4d54-8c95-b017030040ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-2b4285cd-2725-4d54-8c95-b017030040ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:10.272074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:10.273008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:10.273839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 331/670] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:10.274569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:10.274642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4285cd-2725-4d54-8c95-b017030040ee None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:10.310762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1f944-b4a9-497a-9d84-7627d34017d0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:10.344223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf57fb5-0bb3-4ab2-9a70-240d074ff8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:10.344881 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf57fb5-0bb3-4ab2-9a70-240d074ff8dd None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.345163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf57fb5-0bb3-4ab2-9a70-240d074ff8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.345441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf57fb5-0bb3-4ab2-9a70-240d074ff8dd None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.346190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf57fb5-0bb3-4ab2-9a70-240d074ff8dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:10.346497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf57fb5-0bb3-4ab2-9a70-240d074ff8dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:10.347864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf57fb5-0bb3-4ab2-9a70-240d074ff8dd None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:10.359825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 340/671] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:10.361028 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8d4c4-74c6-4b08-b0d8-07e0e28b047c None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.361319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8d4c4-74c6-4b08-b0d8-07e0e28b047c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.361597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8d4c4-74c6-4b08-b0d8-07e0e28b047c None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.362839 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 341/672] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:10.363528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8d4c4-74c6-4b08-b0d8-07e0e28b047c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:10.383359 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4285cd-2725-4d54-8c95-b017030040ee None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:10.393837 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:10.421519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79208f0-b5ac-41a9-82a5-2765a88a05e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:10.422220 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79208f0-b5ac-41a9-82a5-2765a88a05e7 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.422792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79208f0-b5ac-41a9-82a5-2765a88a05e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.422792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79208f0-b5ac-41a9-82a5-2765a88a05e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.423500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79208f0-b5ac-41a9-82a5-2765a88a05e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:10.423810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79208f0-b5ac-41a9-82a5-2765a88a05e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:10.425179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79208f0-b5ac-41a9-82a5-2765a88a05e7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:10.436777 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 332/673] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:10.437568 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79208f0-b5ac-41a9-82a5-2765a88a05e7 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:10.473259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8d4c4-74c6-4b08-b0d8-07e0e28b047c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:10.501079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:10.509083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47348abc-6098-408b-8de0-59fa3eef9728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:10.509680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47348abc-6098-408b-8de0-59fa3eef9728 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.510515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47348abc-6098-408b-8de0-59fa3eef9728 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.510812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47348abc-6098-408b-8de0-59fa3eef9728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.511646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47348abc-6098-408b-8de0-59fa3eef9728 None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=eb4c5027951c4c55bd826943cec15309) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:10.511961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47348abc-6098-408b-8de0-59fa3eef9728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:10.513401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47348abc-6098-408b-8de0-59fa3eef9728 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:10.519190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 342/674] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:10 2022] GET /identity/v3/role_assignments?user.id=eb4c5027951c4c55bd826943cec15309&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:10.519961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47348abc-6098-408b-8de0-59fa3eef9728 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:10.544117 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79208f0-b5ac-41a9-82a5-2765a88a05e7 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:10.581134 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c4df4-964c-4901-9ef5-f443a5707d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:10.581808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c4df4-964c-4901-9ef5-f443a5707d28 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.582124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c4df4-964c-4901-9ef5-f443a5707d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.582399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c4df4-964c-4901-9ef5-f443a5707d28 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.583158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c4df4-964c-4901-9ef5-f443a5707d28 None admin] RBAC: Adding query filter params (name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:10.583462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c4df4-964c-4901-9ef5-f443a5707d28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:10.584848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c4df4-964c-4901-9ef5-f443a5707d28 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:10.593853 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 333/675] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:27:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:10.595209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50cd868-8ffb-4e24-b0b1-6c0be8f9303a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.595491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50cd868-8ffb-4e24-b0b1-6c0be8f9303a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.595765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50cd868-8ffb-4e24-b0b1-6c0be8f9303a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.617550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f50cd868-8ffb-4e24-b0b1-6c0be8f9303a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:10.618788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f50cd868-8ffb-4e24-b0b1-6c0be8f9303a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:10.648791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:10.657592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47348abc-6098-408b-8de0-59fa3eef9728 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:10.700586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 334/676] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:10.701152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e519f-e2b2-43c8-bb96-d816fc306f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:10.701510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50cd868-8ffb-4e24-b0b1-6c0be8f9303a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:10.702835 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e519f-e2b2-43c8-bb96-d816fc306f55 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.702835 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e519f-e2b2-43c8-bb96-d816fc306f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.702835 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e519f-e2b2-43c8-bb96-d816fc306f55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.712705 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e519f-e2b2-43c8-bb96-d816fc306f55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:10.714902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e519f-e2b2-43c8-bb96-d816fc306f55 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a88e519f-e2b2-43c8-bb96-d816fc306f55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:10.732055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:10.733055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:10.734198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:10.734198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:10.734198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:10.734198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:10.734198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:10.734198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:10.734198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:10.734198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:10.734198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 343/677] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:10.734198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e519f-e2b2-43c8-bb96-d816fc306f55 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:10.792585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:10.810845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50cd868-8ffb-4e24-b0b1-6c0be8f9303a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:10.841782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e519f-e2b2-43c8-bb96-d816fc306f55 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:10.848422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624a12c-e0c1-4e48-8c01-e3ce9b0af728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:10.849940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624a12c-e0c1-4e48-8c01-e3ce9b0af728 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.850386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624a12c-e0c1-4e48-8c01-e3ce9b0af728 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.850806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624a12c-e0c1-4e48-8c01-e3ce9b0af728 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.861769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624a12c-e0c1-4e48-8c01-e3ce9b0af728 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:10.863263 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624a12c-e0c1-4e48-8c01-e3ce9b0af728 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-3624a12c-e0c1-4e48-8c01-e3ce9b0af728 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:10.877553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:10.878694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:10.879620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:10.879620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:10.879620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:10.879620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:10.879620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:10.879620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:10.879620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:10.879620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:10.879620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 335/678] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:10.880110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624a12c-e0c1-4e48-8c01-e3ce9b0af728 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:10.888862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd092a1-a721-492e-8cf0-ed7f6714c44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:10.889660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd092a1-a721-492e-8cf0-ed7f6714c44c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:10.889997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd092a1-a721-492e-8cf0-ed7f6714c44c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:10.890278 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd092a1-a721-492e-8cf0-ed7f6714c44c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:10.901557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd092a1-a721-492e-8cf0-ed7f6714c44c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:10.903316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd092a1-a721-492e-8cf0-ed7f6714c44c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:10.907412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-afd092a1-a721-492e-8cf0-ed7f6714c44c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:10.921844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:10.922985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:10.923890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:10.923890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:10.923890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:10.923890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:10.923890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 344/679] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:10.923890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd092a1-a721-492e-8cf0-ed7f6714c44c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:10.997221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624a12c-e0c1-4e48-8c01-e3ce9b0af728 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:11.027491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd092a1-a721-492e-8cf0-ed7f6714c44c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:11.030811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:11.035930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e541df-c671-4e30-85f7-5f7c062be8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:11.036816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e541df-c671-4e30-85f7-5f7c062be8fb None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:11.037059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e541df-c671-4e30-85f7-5f7c062be8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:11.037335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e541df-c671-4e30-85f7-5f7c062be8fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:11.048400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e541df-c671-4e30-85f7-5f7c062be8fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-05e541df-c671-4e30-85f7-5f7c062be8fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:11.055970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:11.056933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:11.057785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 336/680] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:11.059178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e541df-c671-4e30-85f7-5f7c062be8fb None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:11.071820 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edb238-6580-45d9-b021-07a30eaa1cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:11.072486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edb238-6580-45d9-b021-07a30eaa1cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:11.072776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edb238-6580-45d9-b021-07a30eaa1cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:11.073048 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edb238-6580-45d9-b021-07a30eaa1cd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:11.073843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edb238-6580-45d9-b021-07a30eaa1cd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:11.074186 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edb238-6580-45d9-b021-07a30eaa1cd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:11.075624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edb238-6580-45d9-b021-07a30eaa1cd2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:11.087967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 345/681] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:11.089273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edb238-6580-45d9-b021-07a30eaa1cd2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:11.151847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:11.174913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e541df-c671-4e30-85f7-5f7c062be8fb None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:11.202048 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edb238-6580-45d9-b021-07a30eaa1cd2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:11.212662 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46429f-4d85-4423-9b3b-8f7915359cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:11.213981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46429f-4d85-4423-9b3b-8f7915359cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:11.214341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46429f-4d85-4423-9b3b-8f7915359cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:11.214696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46429f-4d85-4423-9b3b-8f7915359cc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:11.215557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46429f-4d85-4423-9b3b-8f7915359cc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:11.215941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46429f-4d85-4423-9b3b-8f7915359cc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:11.217459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46429f-4d85-4423-9b3b-8f7915359cc9 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:11.230414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 337/682] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:11.231472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46429f-4d85-4423-9b3b-8f7915359cc9 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:11.242333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa48771-0f70-470b-a680-8e6990eeb674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:11.242997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa48771-0f70-470b-a680-8e6990eeb674 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:11.243283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa48771-0f70-470b-a680-8e6990eeb674 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:11.243557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa48771-0f70-470b-a680-8e6990eeb674 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:11.244281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa48771-0f70-470b-a680-8e6990eeb674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:11.244585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa48771-0f70-470b-a680-8e6990eeb674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:11.245964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa48771-0f70-470b-a680-8e6990eeb674 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:11.258818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 346/683] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:11.259633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa48771-0f70-470b-a680-8e6990eeb674 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:11.286097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:11.344478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46429f-4d85-4423-9b3b-8f7915359cc9 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:11.372714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa48771-0f70-470b-a680-8e6990eeb674 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:11.380147 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4447b2-0e89-4f28-b4d5-30289837ecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:11.380812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4447b2-0e89-4f28-b4d5-30289837ecd4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:11.381091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4447b2-0e89-4f28-b4d5-30289837ecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:11.381367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4447b2-0e89-4f28-b4d5-30289837ecd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:11.382132 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4447b2-0e89-4f28-b4d5-30289837ecd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:11.382436 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4447b2-0e89-4f28-b4d5-30289837ecd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:11.383782 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4447b2-0e89-4f28-b4d5-30289837ecd4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:11.391853 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 338/684] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:11.392628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4447b2-0e89-4f28-b4d5-30289837ecd4 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:11.395008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:11.414114 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e3718-71fa-4e2c-add6-b575e9de10ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:11.414973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e3718-71fa-4e2c-add6-b575e9de10ca None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:11.415420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e3718-71fa-4e2c-add6-b575e9de10ca None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:11.415849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e3718-71fa-4e2c-add6-b575e9de10ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:11.417365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e3718-71fa-4e2c-add6-b575e9de10ca None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=6b07dcd8c0de4d1694d4363520726df2) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:11.417854 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e3718-71fa-4e2c-add6-b575e9de10ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:11.419481 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e3718-71fa-4e2c-add6-b575e9de10ca None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:11.425751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 347/685] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:11 2022] GET /identity/v3/role_assignments?user.id=6b07dcd8c0de4d1694d4363520726df2&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:11.426617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e3718-71fa-4e2c-add6-b575e9de10ca None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:11.525290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4447b2-0e89-4f28-b4d5-30289837ecd4 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:11.566800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7364f-8e33-427f-9999-059fae8cbf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:11.567487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7364f-8e33-427f-9999-059fae8cbf07 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:11.567770 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7364f-8e33-427f-9999-059fae8cbf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:11.568047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7364f-8e33-427f-9999-059fae8cbf07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:11.568922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7364f-8e33-427f-9999-059fae8cbf07 None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=375d321d6de2494794254e50151c02c4) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:11.569238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7364f-8e33-427f-9999-059fae8cbf07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:11.570764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7364f-8e33-427f-9999-059fae8cbf07 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:11.586083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 339/686] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:11 2022] GET /identity/v3/role_assignments?user.id=375d321d6de2494794254e50151c02c4&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:11.587182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7364f-8e33-427f-9999-059fae8cbf07 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:11.693187 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:11.699041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:11.718903 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e3718-71fa-4e2c-add6-b575e9de10ca None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:11.748337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7364f-8e33-427f-9999-059fae8cbf07 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:11.764786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16b0f5-5818-4671-baf5-5d2742d624ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:11.765601 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16b0f5-5818-4671-baf5-5d2742d624ad None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:11.765946 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16b0f5-5818-4671-baf5-5d2742d624ad None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:11.766179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16b0f5-5818-4671-baf5-5d2742d624ad None admin] PATH_INFO: `/v3/users/swift` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:11.776453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16b0f5-5818-4671-baf5-5d2742d624ad None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:11.777947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16b0f5-5818-4671-baf5-5d2742d624ad None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:11.792803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bbd9d-b4e3-4756-9e44-e80d34e6cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:11.793882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bbd9d-b4e3-4756-9e44-e80d34e6cef5 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:11.794391 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bbd9d-b4e3-4756-9e44-e80d34e6cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:11.794845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bbd9d-b4e3-4756-9e44-e80d34e6cef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-ce16b0f5-5818-4671-baf5-5d2742d624ad None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:11.796177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:11.797097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:11.798251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:11.798251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:11.798251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:11.798251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:11.798251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:11.798251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:11.798251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:11.798251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:11.798251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:27:11.798251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:11.798251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 348/687] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:11.802817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16b0f5-5818-4671-baf5-5d2742d624ad None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:11.812983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bbd9d-b4e3-4756-9e44-e80d34e6cef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:11.814529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bbd9d-b4e3-4756-9e44-e80d34e6cef5 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-314bbd9d-b4e3-4756-9e44-e80d34e6cef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:11.834610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:11.835535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:11.836403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:11.836403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:11.836403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:11.836403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:11.839243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 340/688] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:11.844800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bbd9d-b4e3-4756-9e44-e80d34e6cef5 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:11.943576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16b0f5-5818-4671-baf5-5d2742d624ad None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:12.001237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bbd9d-b4e3-4756-9e44-e80d34e6cef5 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:12.014176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3834bc-f0da-46c9-8cd0-d256d117f833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:12.014856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3834bc-f0da-46c9-8cd0-d256d117f833 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:12.015140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3834bc-f0da-46c9-8cd0-d256d117f833 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:12.015419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3834bc-f0da-46c9-8cd0-d256d117f833 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:12.016268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3834bc-f0da-46c9-8cd0-d256d117f833 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:12.016576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3834bc-f0da-46c9-8cd0-d256d117f833 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:12.018042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3834bc-f0da-46c9-8cd0-d256d117f833 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:12.037226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:12.055748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 349/689] 10.222.0.60 () {56 vars in 1135 bytes} [Mon Jul 25 12:27:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:12.061123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3834bc-f0da-46c9-8cd0-d256d117f833 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:12.070245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:12.086725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933856c5-931d-4c84-a351-f69f33ff29dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:12.087686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933856c5-931d-4c84-a351-f69f33ff29dd None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:12.087974 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933856c5-931d-4c84-a351-f69f33ff29dd None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:12.088811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933856c5-931d-4c84-a351-f69f33ff29dd None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:12.090242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933856c5-931d-4c84-a351-f69f33ff29dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:12.090565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933856c5-931d-4c84-a351-f69f33ff29dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:12.092013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933856c5-931d-4c84-a351-f69f33ff29dd None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:12.108474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 341/690] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:12.113217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933856c5-931d-4c84-a351-f69f33ff29dd None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:12.203266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3834bc-f0da-46c9-8cd0-d256d117f833 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:12.258952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2b6a0-6aba-4d1d-8b9e-1d1541ed7aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:12.260865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2b6a0-6aba-4d1d-8b9e-1d1541ed7aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:12.261361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2b6a0-6aba-4d1d-8b9e-1d1541ed7aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:12.262249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2b6a0-6aba-4d1d-8b9e-1d1541ed7aa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:12.353131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933856c5-931d-4c84-a351-f69f33ff29dd None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:12.393804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2b6a0-6aba-4d1d-8b9e-1d1541ed7aa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:12.396766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2b6a0-6aba-4d1d-8b9e-1d1541ed7aa1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:12.410282 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:12.419530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d23b1-1216-430f-9c2e-2cef969887f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:12.420677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d23b1-1216-430f-9c2e-2cef969887f1 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:12.421553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d23b1-1216-430f-9c2e-2cef969887f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:12.422902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d23b1-1216-430f-9c2e-2cef969887f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:12.435711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d23b1-1216-430f-9c2e-2cef969887f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:12.441036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d23b1-1216-430f-9c2e-2cef969887f1 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-762d23b1-1216-430f-9c2e-2cef969887f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:12.461731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:12.462722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:12.463769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:12.463769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:12.463769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:12.463769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:12.463769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:12.463769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:12.463769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:12.463769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:12.463769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:12.463769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:12.476734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 342/691] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Jul 25 12:27:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:12.478845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d23b1-1216-430f-9c2e-2cef969887f1 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:12.548624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:12.607010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 350/692] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:27:12.614065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2b6a0-6aba-4d1d-8b9e-1d1541ed7aa1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:12.631235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d23b1-1216-430f-9c2e-2cef969887f1 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:12.685003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e42a55-64aa-467f-8562-794cb72eed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:12.685887 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e42a55-64aa-467f-8562-794cb72eed25 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:12.686241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e42a55-64aa-467f-8562-794cb72eed25 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:12.686516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e42a55-64aa-467f-8562-794cb72eed25 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:12.687736 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e42a55-64aa-467f-8562-794cb72eed25 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:12.688267 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e42a55-64aa-467f-8562-794cb72eed25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:12.689629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e42a55-64aa-467f-8562-794cb72eed25 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:12.712036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 343/693] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Jul 25 12:27:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:12.718018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421afbc-ed3d-4a8e-89df-7297826552fd None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:12.718544 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421afbc-ed3d-4a8e-89df-7297826552fd None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:12.719016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421afbc-ed3d-4a8e-89df-7297826552fd None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:12.720726 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 344/694] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:12.721621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421afbc-ed3d-4a8e-89df-7297826552fd None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:12.747918 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2b6a0-6aba-4d1d-8b9e-1d1541ed7aa1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:12.755588 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:12.782167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0517dd6-0d8a-4b13-847f-f6a0860c4e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:12.783082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0517dd6-0d8a-4b13-847f-f6a0860c4e5e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:12.783457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0517dd6-0d8a-4b13-847f-f6a0860c4e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:12.783832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0517dd6-0d8a-4b13-847f-f6a0860c4e5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:12.794539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0517dd6-0d8a-4b13-847f-f6a0860c4e5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:12.796094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0517dd6-0d8a-4b13-847f-f6a0860c4e5e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-f0517dd6-0d8a-4b13-847f-f6a0860c4e5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:12.814554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:12.815623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:12.816697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:12.816697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:12.816697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:12.816697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:12.818229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 351/695] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:12.819551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5a384-132b-4da3-a3dc-610cba666353 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:12.819927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5a384-132b-4da3-a3dc-610cba666353 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:12.820292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5a384-132b-4da3-a3dc-610cba666353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:12.838108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-52b5a384-132b-4da3-a3dc-610cba666353 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:12.838295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:12.839417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-52b5a384-132b-4da3-a3dc-610cba666353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:12.840430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421afbc-ed3d-4a8e-89df-7297826552fd None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:12.878830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd005287-8864-4e70-ad3b-6fa30bc1076d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:12.879811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd005287-8864-4e70-ad3b-6fa30bc1076d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:12.880284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd005287-8864-4e70-ad3b-6fa30bc1076d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:12.880740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd005287-8864-4e70-ad3b-6fa30bc1076d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:12.918206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 352/696] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:12.919762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5a384-132b-4da3-a3dc-610cba666353 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:12.955873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd005287-8864-4e70-ad3b-6fa30bc1076d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:12.957718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd005287-8864-4e70-ad3b-6fa30bc1076d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:13.037433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:13.055115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5a384-132b-4da3-a3dc-610cba666353 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:13.066554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:13.067518 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 345/697] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:27:13.068427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd005287-8864-4e70-ad3b-6fa30bc1076d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:13.098473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d34e7e-fe80-4799-93d4-50e20b768288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:13.099816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d34e7e-fe80-4799-93d4-50e20b768288 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:13.100545 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d34e7e-fe80-4799-93d4-50e20b768288 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:13.101010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d34e7e-fe80-4799-93d4-50e20b768288 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:13.102012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d34e7e-fe80-4799-93d4-50e20b768288 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:13.102656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d34e7e-fe80-4799-93d4-50e20b768288 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:13.105472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d34e7e-fe80-4799-93d4-50e20b768288 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:13.124888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 353/698] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:13.124888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d34e7e-fe80-4799-93d4-50e20b768288 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:13.180355 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd005287-8864-4e70-ad3b-6fa30bc1076d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:13.222583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf0429-26e6-4d9d-84ac-e44c769d715c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:13.223838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf0429-26e6-4d9d-84ac-e44c769d715c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:13.224318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf0429-26e6-4d9d-84ac-e44c769d715c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:13.224814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf0429-26e6-4d9d-84ac-e44c769d715c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:13.225865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf0429-26e6-4d9d-84ac-e44c769d715c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:13.227510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf0429-26e6-4d9d-84ac-e44c769d715c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:13.238527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d34e7e-fe80-4799-93d4-50e20b768288 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-5baf0429-26e6-4d9d-84ac-e44c769d715c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:13.247915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:13.248883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:27:13.249775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:13.252936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 346/699] 10.222.0.60 () {56 vars in 1129 bytes} [Mon Jul 25 12:27:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:13.253938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf0429-26e6-4d9d-84ac-e44c769d715c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:13.278150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3278241-4c32-4ee1-a90a-f3e66b390ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:13.279239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3278241-4c32-4ee1-a90a-f3e66b390ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:13.279239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3278241-4c32-4ee1-a90a-f3e66b390ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:13.279449 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3278241-4c32-4ee1-a90a-f3e66b390ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:13.291008 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3278241-4c32-4ee1-a90a-f3e66b390ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:13.293017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3278241-4c32-4ee1-a90a-f3e66b390ba6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:13.301137 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b3278241-4c32-4ee1-a90a-f3e66b390ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:13.312296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:13.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:13.314157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:13.314157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:13.314157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:13.314157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:13.318037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 354/700] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:13.318946 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3278241-4c32-4ee1-a90a-f3e66b390ba6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:13.325939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:13.372283 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf0429-26e6-4d9d-84ac-e44c769d715c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:13.410017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221447c2-e2dc-4156-aea0-455995324b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:13.411290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221447c2-e2dc-4156-aea0-455995324b09 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:13.411802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221447c2-e2dc-4156-aea0-455995324b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:13.412278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221447c2-e2dc-4156-aea0-455995324b09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:13.420074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221447c2-e2dc-4156-aea0-455995324b09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:13.421979 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221447c2-e2dc-4156-aea0-455995324b09 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:13.425822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3278241-4c32-4ee1-a90a-f3e66b390ba6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-221447c2-e2dc-4156-aea0-455995324b09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:13.438167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:13.439156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:13.440019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:13.440019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:13.440019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:13.440019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:13.440019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:13.440019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:13.440019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:13.440019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:13.446192 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 347/701] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:13.447942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221447c2-e2dc-4156-aea0-455995324b09 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:13.461252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356ec37-006c-49c2-97e8-e1fa9637c695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:13.461886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356ec37-006c-49c2-97e8-e1fa9637c695 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:13.462218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356ec37-006c-49c2-97e8-e1fa9637c695 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:13.462488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356ec37-006c-49c2-97e8-e1fa9637c695 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:13.463849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356ec37-006c-49c2-97e8-e1fa9637c695 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:13.464426 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356ec37-006c-49c2-97e8-e1fa9637c695 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:13.465600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356ec37-006c-49c2-97e8-e1fa9637c695 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:13.470801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 355/702] 10.222.0.60 () {56 vars in 1124 bytes} [Mon Jul 25 12:27:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:13.471535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356ec37-006c-49c2-97e8-e1fa9637c695 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:13.543443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:13.560461 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221447c2-e2dc-4156-aea0-455995324b09 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:13.583160 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:13.586169 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356ec37-006c-49c2-97e8-e1fa9637c695 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:13.598934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06309a74-72b8-4aac-8dc9-be77e1aa0c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:13.599645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06309a74-72b8-4aac-8dc9-be77e1aa0c53 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:13.600064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06309a74-72b8-4aac-8dc9-be77e1aa0c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:13.600446 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06309a74-72b8-4aac-8dc9-be77e1aa0c53 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:13.601215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06309a74-72b8-4aac-8dc9-be77e1aa0c53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:13.601623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06309a74-72b8-4aac-8dc9-be77e1aa0c53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:13.603436 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06309a74-72b8-4aac-8dc9-be77e1aa0c53 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:13.620308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 348/703] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:13.620308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06309a74-72b8-4aac-8dc9-be77e1aa0c53 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:13.629441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49143501-51bb-41e8-8d5c-30bc0dd1a002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:13.630423 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49143501-51bb-41e8-8d5c-30bc0dd1a002 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:13.630911 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49143501-51bb-41e8-8d5c-30bc0dd1a002 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:13.631367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49143501-51bb-41e8-8d5c-30bc0dd1a002 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:13.632385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49143501-51bb-41e8-8d5c-30bc0dd1a002 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:13.632971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49143501-51bb-41e8-8d5c-30bc0dd1a002 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:13.634693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49143501-51bb-41e8-8d5c-30bc0dd1a002 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:13.648972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 356/704] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:13.650117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49143501-51bb-41e8-8d5c-30bc0dd1a002 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:13.739173 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06309a74-72b8-4aac-8dc9-be77e1aa0c53 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:13.761235 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49143501-51bb-41e8-8d5c-30bc0dd1a002 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:13.777565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afb041-3c5b-4d54-9e8b-b2f5292729ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:13.778607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afb041-3c5b-4d54-9e8b-b2f5292729ce None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:13.778966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afb041-3c5b-4d54-9e8b-b2f5292729ce None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:13.779308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afb041-3c5b-4d54-9e8b-b2f5292729ce None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:13.780342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afb041-3c5b-4d54-9e8b-b2f5292729ce None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:13.780737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afb041-3c5b-4d54-9e8b-b2f5292729ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:13.782615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afb041-3c5b-4d54-9e8b-b2f5292729ce None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:13.788398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:13.796981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 349/705] 10.222.0.60 () {56 vars in 1124 bytes} [Mon Jul 25 12:27:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:13.796981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afb041-3c5b-4d54-9e8b-b2f5292729ce None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:13.814173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58666df-e132-44c2-ae3a-f8f7344e7435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:13.815141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58666df-e132-44c2-ae3a-f8f7344e7435 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:13.815424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58666df-e132-44c2-ae3a-f8f7344e7435 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:13.815789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58666df-e132-44c2-ae3a-f8f7344e7435 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:13.824879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58666df-e132-44c2-ae3a-f8f7344e7435 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:13.826526 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58666df-e132-44c2-ae3a-f8f7344e7435 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b58666df-e132-44c2-ae3a-f8f7344e7435 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:13.842819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:13.843766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:13.844654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:13.844654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:13.844654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:13.844654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:13.844654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:13.844654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:13.844654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:13.844654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:13.844654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 357/706] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:13.845952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc1915-12d4-4d8e-a6e7-4801590a3bd5 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:13.846814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:13.847931 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc1915-12d4-4d8e-a6e7-4801590a3bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:13.847931 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc1915-12d4-4d8e-a6e7-4801590a3bd5 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:13.853149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 358/707] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:13.854954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc1915-12d4-4d8e-a6e7-4801590a3bd5 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:13.912392 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afb041-3c5b-4d54-9e8b-b2f5292729ce None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:13.950455 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430098aa-6c69-4133-b992-1fb866fdfa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:13.951120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430098aa-6c69-4133-b992-1fb866fdfa65 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:13.951407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430098aa-6c69-4133-b992-1fb866fdfa65 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:13.951684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430098aa-6c69-4133-b992-1fb866fdfa65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:13.952535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430098aa-6c69-4133-b992-1fb866fdfa65 None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=e56b0d48e08c4f778a7c509c10edc871) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:13.952849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430098aa-6c69-4133-b992-1fb866fdfa65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:13.954330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430098aa-6c69-4133-b992-1fb866fdfa65 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:13.969397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 350/708] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:13 2022] GET /identity/v3/role_assignments?user.id=e56b0d48e08c4f778a7c509c10edc871&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:13.969397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266787d1-6b17-4088-9f72-1bbb8e06d712 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:13.969397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266787d1-6b17-4088-9f72-1bbb8e06d712 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:13.969397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266787d1-6b17-4088-9f72-1bbb8e06d712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:13.984765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-266787d1-6b17-4088-9f72-1bbb8e06d712 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:13.985150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-266787d1-6b17-4088-9f72-1bbb8e06d712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:13.985712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc1915-12d4-4d8e-a6e7-4801590a3bd5 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:14.035149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487be08-60e8-456e-9a9c-7c11062aa238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:14.035827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487be08-60e8-456e-9a9c-7c11062aa238 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:14.036106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487be08-60e8-456e-9a9c-7c11062aa238 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:14.036379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487be08-60e8-456e-9a9c-7c11062aa238 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:14.037161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487be08-60e8-456e-9a9c-7c11062aa238 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:14.038027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487be08-60e8-456e-9a9c-7c11062aa238 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:14.040964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487be08-60e8-456e-9a9c-7c11062aa238 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:14.057549 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 359/709] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:14.059776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660821ef-d83b-4e54-8e71-94ea051a2756 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:14.060179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660821ef-d83b-4e54-8e71-94ea051a2756 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:14.060455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660821ef-d83b-4e54-8e71-94ea051a2756 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:14.062892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 360/710] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:14.064846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660821ef-d83b-4e54-8e71-94ea051a2756 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:14.075993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 351/711] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:14.077340 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d138d-3307-4cf6-a1db-2c994634ac4e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:14.077625 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d138d-3307-4cf6-a1db-2c994634ac4e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:14.077899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d138d-3307-4cf6-a1db-2c994634ac4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:14.099543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-170d138d-3307-4cf6-a1db-2c994634ac4e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:14.100804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-170d138d-3307-4cf6-a1db-2c994634ac4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:14.143964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:14.154962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:14.195832 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 352/712] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:14.196682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d138d-3307-4cf6-a1db-2c994634ac4e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:14.223878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660821ef-d83b-4e54-8e71-94ea051a2756 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:14.271834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef32585-0940-4bb5-bb16-e3974cfb88a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:14.272482 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef32585-0940-4bb5-bb16-e3974cfb88a6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:14.272756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef32585-0940-4bb5-bb16-e3974cfb88a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:14.273027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef32585-0940-4bb5-bb16-e3974cfb88a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:14.273941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef32585-0940-4bb5-bb16-e3974cfb88a6 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=ccf2dedaf7a449d2b51b9e9f3b420091) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:14.274249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef32585-0940-4bb5-bb16-e3974cfb88a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:14.276063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef32585-0940-4bb5-bb16-e3974cfb88a6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:14.283559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 361/713] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:14 2022] GET /identity/v3/role_assignments?user.id=ccf2dedaf7a449d2b51b9e9f3b420091&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:14.284499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef32585-0940-4bb5-bb16-e3974cfb88a6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:14.326661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d138d-3307-4cf6-a1db-2c994634ac4e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:14.373568 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d6044-abc8-4f9e-b847-744c2acdd142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:14.374731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d6044-abc8-4f9e-b847-744c2acdd142 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:14.375195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d6044-abc8-4f9e-b847-744c2acdd142 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:14.375645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d6044-abc8-4f9e-b847-744c2acdd142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:14.390899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d6044-abc8-4f9e-b847-744c2acdd142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:14.392741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d6044-abc8-4f9e-b847-744c2acdd142 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:14.408144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef32585-0940-4bb5-bb16-e3974cfb88a6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-1c8d6044-abc8-4f9e-b847-744c2acdd142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:14.417508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:14.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:14.419488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:14.419488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:14.419488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:14.419488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:14.419710 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:14.424982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 353/714] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:14.431115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d6044-abc8-4f9e-b847-744c2acdd142 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:14.489888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34574876-3010-47d0-819a-8fa0f9142079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:14.493049 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34574876-3010-47d0-819a-8fa0f9142079 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:14.494293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34574876-3010-47d0-819a-8fa0f9142079 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:14.495030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:14.495770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34574876-3010-47d0-819a-8fa0f9142079 None admin] PATH_INFO: `/v3/services/network` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:14.496997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34574876-3010-47d0-819a-8fa0f9142079 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:14.499375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34574876-3010-47d0-819a-8fa0f9142079 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-34574876-3010-47d0-819a-8fa0f9142079 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:14.530120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:14.531206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:27:14.532057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:14.541348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 362/715] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:14 2022] GET /identity/v3/services/network => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:14.546467 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34574876-3010-47d0-819a-8fa0f9142079 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:14.724423 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d6044-abc8-4f9e-b847-744c2acdd142 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:14.781737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de50d78-a1e4-4668-98c4-41fd2d1331be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:14.782441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de50d78-a1e4-4668-98c4-41fd2d1331be None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:14.782725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de50d78-a1e4-4668-98c4-41fd2d1331be None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:14.783000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de50d78-a1e4-4668-98c4-41fd2d1331be None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:14.783731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de50d78-a1e4-4668-98c4-41fd2d1331be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:14.784039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de50d78-a1e4-4668-98c4-41fd2d1331be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:14.785634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de50d78-a1e4-4668-98c4-41fd2d1331be None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:14.794821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34574876-3010-47d0-819a-8fa0f9142079 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:14.799431 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 354/716] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:14.804605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de50d78-a1e4-4668-98c4-41fd2d1331be None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:14.919340 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:14.944847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de50d78-a1e4-4668-98c4-41fd2d1331be None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:14.946618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccca6a-7132-4ab1-9e3a-18a59e0c6706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:14.947303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccca6a-7132-4ab1-9e3a-18a59e0c6706 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:14.947616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccca6a-7132-4ab1-9e3a-18a59e0c6706 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:14.947894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccca6a-7132-4ab1-9e3a-18a59e0c6706 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:14.948607 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ccca6a-7132-4ab1-9e3a-18a59e0c6706 None admin] RBAC: Adding query filter params (name=network) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:14.948912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ccca6a-7132-4ab1-9e3a-18a59e0c6706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:14.956279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ccca6a-7132-4ab1-9e3a-18a59e0c6706 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:14.982478 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 363/717] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:14.983384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccca6a-7132-4ab1-9e3a-18a59e0c6706 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:15.036548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12239d1b-d3b3-4fcb-a1e3-a094ca05ebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:15.037231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239d1b-d3b3-4fcb-a1e3-a094ca05ebb7 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:15.037512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239d1b-d3b3-4fcb-a1e3-a094ca05ebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:15.037786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239d1b-d3b3-4fcb-a1e3-a094ca05ebb7 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:15.043227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12239d1b-d3b3-4fcb-a1e3-a094ca05ebb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:15.044684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12239d1b-d3b3-4fcb-a1e3-a094ca05ebb7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:15.078974 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:15.099422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 355/718] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:27:14 2022] POST /identity/v3/users => generated 254 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:27:15.198009 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccca6a-7132-4ab1-9e3a-18a59e0c6706 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:15.236310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a1362-7867-4b06-9b2c-4aa0b04d6103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:15.236904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a1362-7867-4b06-9b2c-4aa0b04d6103 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:15.237185 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a1362-7867-4b06-9b2c-4aa0b04d6103 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:15.237513 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a1362-7867-4b06-9b2c-4aa0b04d6103 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:15.238261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a1362-7867-4b06-9b2c-4aa0b04d6103 None admin] RBAC: Adding query filter params (type=network) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:15.238616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a1362-7867-4b06-9b2c-4aa0b04d6103 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:15.239992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a1362-7867-4b06-9b2c-4aa0b04d6103 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:15.246855 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 364/719] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:16.250686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68878e1c-caa5-4dc6-a64e-2f8d63af1d8a None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:16.250686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68878e1c-caa5-4dc6-a64e-2f8d63af1d8a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:16.250686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68878e1c-caa5-4dc6-a64e-2f8d63af1d8a None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:16.252254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 356/720] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:16.257870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004de81-5949-4320-b012-2e1af929fd3c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:16.258330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004de81-5949-4320-b012-2e1af929fd3c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:16.258533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004de81-5949-4320-b012-2e1af929fd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:16.286889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f004de81-5949-4320-b012-2e1af929fd3c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:16.288086 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f004de81-5949-4320-b012-2e1af929fd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:16.380343 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 365/721] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:27:16.416815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68878e1c-caa5-4dc6-a64e-2f8d63af1d8a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:16.470825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b149c-606d-466d-abc7-83d458d7be4f None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:16.471120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b149c-606d-466d-abc7-83d458d7be4f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:16.471314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b149c-606d-466d-abc7-83d458d7be4f None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:16.472956 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 366/722] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:16.489372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d8426-b1ca-4765-93ae-93ec8ffd2fee None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:16.489857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d8426-b1ca-4765-93ae-93ec8ffd2fee None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:16.490006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d8426-b1ca-4765-93ae-93ec8ffd2fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:16.520957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4a0d8426-b1ca-4765-93ae-93ec8ffd2fee None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:16.522182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4a0d8426-b1ca-4765-93ae-93ec8ffd2fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:16.575066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68878e1c-caa5-4dc6-a64e-2f8d63af1d8a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:16.575720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:16.621330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448e6b6-94b7-4a88-b205-ea657f90856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:16.622412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448e6b6-94b7-4a88-b205-ea657f90856a None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:16.622911 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448e6b6-94b7-4a88-b205-ea657f90856a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:16.623406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448e6b6-94b7-4a88-b205-ea657f90856a None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:16.624329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448e6b6-94b7-4a88-b205-ea657f90856a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:16.625942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448e6b6-94b7-4a88-b205-ea657f90856a None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:16.631765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:16.654217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 357/723] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:27:16 2022] POST /identity/v3/services => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:27:16.673285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 367/724] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:27:16.697198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448e6b6-94b7-4a88-b205-ea657f90856a None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:16.833174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448e6b6-94b7-4a88-b205-ea657f90856a None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:16.884567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9da464-d8ae-4896-9632-9a855ac55103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:16.885439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9da464-d8ae-4896-9632-9a855ac55103 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:16.885723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9da464-d8ae-4896-9632-9a855ac55103 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:16.886105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9da464-d8ae-4896-9632-9a855ac55103 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:16.897138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9da464-d8ae-4896-9632-9a855ac55103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-eb9da464-d8ae-4896-9632-9a855ac55103 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:16.905417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:16.906407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:16.907274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 358/725] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:16.916103 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d8426-b1ca-4765-93ae-93ec8ffd2fee None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:17.032996 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d8426-b1ca-4765-93ae-93ec8ffd2fee None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:17.051988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28b7ed-6cb5-480d-9520-a9d0abb654ba None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:17.052284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28b7ed-6cb5-480d-9520-a9d0abb654ba None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:17.052537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28b7ed-6cb5-480d-9520-a9d0abb654ba None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:17.054175 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 359/726] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:17.059236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be7a69-868d-4654-a4d3-307ad7f16eef None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:17.059523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be7a69-868d-4654-a4d3-307ad7f16eef None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:17.059805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be7a69-868d-4654-a4d3-307ad7f16eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:17.076522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004da543-e7cc-45d5-aca0-4360c0546bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:17.077761 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004da543-e7cc-45d5-aca0-4360c0546bca None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:17.078364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004da543-e7cc-45d5-aca0-4360c0546bca None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:17.078830 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004da543-e7cc-45d5-aca0-4360c0546bca None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:17.079788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004da543-e7cc-45d5-aca0-4360c0546bca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:17.080333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004da543-e7cc-45d5-aca0-4360c0546bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:17.080997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-00be7a69-868d-4654-a4d3-307ad7f16eef None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:17.082118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004da543-e7cc-45d5-aca0-4360c0546bca None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:17.082368 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-00be7a69-868d-4654-a4d3-307ad7f16eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:17.093387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 368/727] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:17.105080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004da543-e7cc-45d5-aca0-4360c0546bca None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:17.151583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:17.162049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:17.180639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 360/728] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:27:17.218412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be7a69-868d-4654-a4d3-307ad7f16eef None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:17.301618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004da543-e7cc-45d5-aca0-4360c0546bca None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:17.372640 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d3a3a-77ea-4762-adba-0b93cd85703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:17.373363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d3a3a-77ea-4762-adba-0b93cd85703c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:17.373644 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d3a3a-77ea-4762-adba-0b93cd85703c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:17.373939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d3a3a-77ea-4762-adba-0b93cd85703c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:17.392347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d3a3a-77ea-4762-adba-0b93cd85703c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:17.393821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d3a3a-77ea-4762-adba-0b93cd85703c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:17.407583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be7a69-868d-4654-a4d3-307ad7f16eef None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-f09d3a3a-77ea-4762-adba-0b93cd85703c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:17.413202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:17.414134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:17.415057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:17.415057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:17.415057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:17.415057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:17.415057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 369/729] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:17.420484 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d3a3a-77ea-4762-adba-0b93cd85703c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:17.472075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16f9c0-10a9-475b-8e3e-e8d63764853e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:17.472929 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16f9c0-10a9-475b-8e3e-e8d63764853e None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:17.473206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16f9c0-10a9-475b-8e3e-e8d63764853e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:17.473479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16f9c0-10a9-475b-8e3e-e8d63764853e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:17.496094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16f9c0-10a9-475b-8e3e-e8d63764853e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-ef16f9c0-10a9-475b-8e3e-e8d63764853e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:17.504659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:17.505602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:17.506492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:17.506492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:17.506492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:17.506492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:17.506492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:17.506492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:17.506492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:17.506492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:17.506492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:17.506492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:17.506492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:17.506492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:17.511841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 361/730] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:17.515440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16f9c0-10a9-475b-8e3e-e8d63764853e None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:17.546292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:17.597237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:17.603201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d3a3a-77ea-4762-adba-0b93cd85703c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:17.642046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a90f9-d2ba-4105-a2ec-163a6626939b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:17.642885 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a90f9-d2ba-4105-a2ec-163a6626939b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:17.643273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a90f9-d2ba-4105-a2ec-163a6626939b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:17.643676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a90f9-d2ba-4105-a2ec-163a6626939b None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:17.645143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a90f9-d2ba-4105-a2ec-163a6626939b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:17.645563 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a90f9-d2ba-4105-a2ec-163a6626939b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:17.647081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a90f9-d2ba-4105-a2ec-163a6626939b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:17.661374 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16f9c0-10a9-475b-8e3e-e8d63764853e None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:17.669937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 370/731] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:17.669937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb889fd-17de-4faa-a3e1-8907991c8943 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:17.669937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb889fd-17de-4faa-a3e1-8907991c8943 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:17.669937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb889fd-17de-4faa-a3e1-8907991c8943 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:17.669937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 371/732] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:17.673390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb889fd-17de-4faa-a3e1-8907991c8943 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:17.707143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24a28b-4946-4d5c-a5da-37781df65bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:17.708464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24a28b-4946-4d5c-a5da-37781df65bac None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:17.708957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24a28b-4946-4d5c-a5da-37781df65bac None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:17.709683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24a28b-4946-4d5c-a5da-37781df65bac None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:17.711601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24a28b-4946-4d5c-a5da-37781df65bac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:17.712116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24a28b-4946-4d5c-a5da-37781df65bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:17.714623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24a28b-4946-4d5c-a5da-37781df65bac None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:17.730169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 362/733] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:17.731727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376a181-274d-485c-97a7-580ae3daeb53 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:17.732077 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376a181-274d-485c-97a7-580ae3daeb53 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:17.732621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376a181-274d-485c-97a7-580ae3daeb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:17.752683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-2376a181-274d-485c-97a7-580ae3daeb53 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:17.754007 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2376a181-274d-485c-97a7-580ae3daeb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:17.808457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb889fd-17de-4faa-a3e1-8907991c8943 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:17.839965 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 363/734] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:27:17.839965 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376a181-274d-485c-97a7-580ae3daeb53 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:17.855486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:17.865068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848f2d5-9807-4259-922f-34515d45d5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:17.866274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848f2d5-9807-4259-922f-34515d45d5da None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:17.866710 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848f2d5-9807-4259-922f-34515d45d5da None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:17.867307 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848f2d5-9807-4259-922f-34515d45d5da None admin] PATH_INFO: `/v3/users/swift` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:17.876410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848f2d5-9807-4259-922f-34515d45d5da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:17.878132 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848f2d5-9807-4259-922f-34515d45d5da None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:17.891861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b848f2d5-9807-4259-922f-34515d45d5da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:17.894291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:17.895206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:17.896144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:17.896144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:17.896144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:17.896144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:17.896144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:17.896144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:17.896144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:17.896144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:17.896144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:27:17.896144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:17.896144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 372/735] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:17.896728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848f2d5-9807-4259-922f-34515d45d5da None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:17.966577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376a181-274d-485c-97a7-580ae3daeb53 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:18.000653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc3cfb-6c30-42e9-9919-0514cc00b0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:18.001430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc3cfb-6c30-42e9-9919-0514cc00b0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:18.001709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc3cfb-6c30-42e9-9919-0514cc00b0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:18.002013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc3cfb-6c30-42e9-9919-0514cc00b0b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:18.005943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848f2d5-9807-4259-922f-34515d45d5da None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:18.012513 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc3cfb-6c30-42e9-9919-0514cc00b0b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:18.013961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc3cfb-6c30-42e9-9919-0514cc00b0b6 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-dddc3cfb-6c30-42e9-9919-0514cc00b0b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:18.031145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:18.032217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:18.037592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:18.037592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:18.037592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:18.037592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:18.037592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 364/736] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:18.037592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec75991-a500-4f61-9d44-2daf172f8505 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:18.037592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec75991-a500-4f61-9d44-2daf172f8505 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:18.037592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec75991-a500-4f61-9d44-2daf172f8505 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:18.037592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 365/737] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:18.038208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec75991-a500-4f61-9d44-2daf172f8505 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:18.049446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505fb75-862d-420b-8627-8b480b0e4056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:18.050143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505fb75-862d-420b-8627-8b480b0e4056 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:18.050428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505fb75-862d-420b-8627-8b480b0e4056 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:18.050701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505fb75-862d-420b-8627-8b480b0e4056 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:18.051430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505fb75-862d-420b-8627-8b480b0e4056 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:18.052788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505fb75-862d-420b-8627-8b480b0e4056 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:18.068922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 373/738] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:27:17 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:27:18.071333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505fb75-862d-420b-8627-8b480b0e4056 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:18.143379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:18.156935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:18.199559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec75991-a500-4f61-9d44-2daf172f8505 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:18.227064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505fb75-862d-420b-8627-8b480b0e4056 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:18.249146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5a1cb-2d2c-444b-a462-4a01a684d0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:18.249831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5a1cb-2d2c-444b-a462-4a01a684d0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:18.250141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5a1cb-2d2c-444b-a462-4a01a684d0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:18.250418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5a1cb-2d2c-444b-a462-4a01a684d0f0 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:18.251270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5a1cb-2d2c-444b-a462-4a01a684d0f0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:18.251726 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5a1cb-2d2c-444b-a462-4a01a684d0f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:18.253194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5a1cb-2d2c-444b-a462-4a01a684d0f0 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:18.269354 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6480174-d2b2-4ff2-ba57-b61bc5b30373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:18.269695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6480174-d2b2-4ff2-ba57-b61bc5b30373 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:18.270042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6480174-d2b2-4ff2-ba57-b61bc5b30373 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:18.270320 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6480174-d2b2-4ff2-ba57-b61bc5b30373 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:18.271052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6480174-d2b2-4ff2-ba57-b61bc5b30373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:18.271410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6480174-d2b2-4ff2-ba57-b61bc5b30373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:18.272722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6480174-d2b2-4ff2-ba57-b61bc5b30373 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:18.274097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 366/739] 10.222.0.60 () {56 vars in 1135 bytes} [Mon Jul 25 12:27:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:18.279403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf2e7e-c4c8-44a8-a7cf-b00a687bd73b None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:18.279917 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf2e7e-c4c8-44a8-a7cf-b00a687bd73b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:18.280389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf2e7e-c4c8-44a8-a7cf-b00a687bd73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:18.286818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 374/740] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:18.287640 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6480174-d2b2-4ff2-ba57-b61bc5b30373 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:18.303637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-5aaf2e7e-c4c8-44a8-a7cf-b00a687bd73b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:18.304862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5aaf2e7e-c4c8-44a8-a7cf-b00a687bd73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:18.401161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 367/741] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:18.402370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf2e7e-c4c8-44a8-a7cf-b00a687bd73b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:18.421037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6480174-d2b2-4ff2-ba57-b61bc5b30373 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:18.439230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:18.468274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9f26b-587d-4a54-9c82-a2cd88c034e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:18.469317 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9f26b-587d-4a54-9c82-a2cd88c034e9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:18.469857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9f26b-587d-4a54-9c82-a2cd88c034e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:18.470306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9f26b-587d-4a54-9c82-a2cd88c034e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:18.526391 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:18.554653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9f26b-587d-4a54-9c82-a2cd88c034e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:18.556853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9f26b-587d-4a54-9c82-a2cd88c034e9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:18.561478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf2e7e-c4c8-44a8-a7cf-b00a687bd73b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:18.602096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22d525-5eac-497a-a506-b4c7bfed9482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:18.603167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d525-5eac-497a-a506-b4c7bfed9482 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:18.603661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d525-5eac-497a-a506-b4c7bfed9482 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:18.604077 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d525-5eac-497a-a506-b4c7bfed9482 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:18.612674 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22d525-5eac-497a-a506-b4c7bfed9482 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:18.614372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22d525-5eac-497a-a506-b4c7bfed9482 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-7c22d525-5eac-497a-a506-b4c7bfed9482 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:18.630764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:18.631926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:18.632988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:18.632988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:18.632988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:18.632988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:18.632988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:18.632988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:18.632988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:18.632988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:18.632988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:18.632988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:18.635764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 368/742] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Jul 25 12:27:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:18.636674 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22d525-5eac-497a-a506-b4c7bfed9482 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:18.702232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 375/743] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:27:18.703886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9f26b-587d-4a54-9c82-a2cd88c034e9 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:18.719516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:18.775939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22d525-5eac-497a-a506-b4c7bfed9482 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:18.817411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9f26b-587d-4a54-9c82-a2cd88c034e9 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:18.824873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175ebbe-0032-4044-92a4-e270765d4399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:18.825679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175ebbe-0032-4044-92a4-e270765d4399 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:18.825982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175ebbe-0032-4044-92a4-e270765d4399 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:18.826261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175ebbe-0032-4044-92a4-e270765d4399 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:18.828718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:18.840854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175ebbe-0032-4044-92a4-e270765d4399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-7175ebbe-0032-4044-92a4-e270765d4399 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:18.851816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:18.852973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:18.854486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:18.854486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:18.854486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:18.854486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:18.854486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:18.854486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:18.854486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:18.854486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:18.854486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:18.854486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:18.854486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:18.854486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:18.857994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 369/744] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:18.858987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175ebbe-0032-4044-92a4-e270765d4399 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:18.865393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ba9bd-7e85-419a-9f1f-4e0721940770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:18.866382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ba9bd-7e85-419a-9f1f-4e0721940770 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:18.866866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ba9bd-7e85-419a-9f1f-4e0721940770 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:18.867316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ba9bd-7e85-419a-9f1f-4e0721940770 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:18.868391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ba9bd-7e85-419a-9f1f-4e0721940770 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:18.868916 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ba9bd-7e85-419a-9f1f-4e0721940770 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:18.871320 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ba9bd-7e85-419a-9f1f-4e0721940770 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:18.893310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 376/745] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Jul 25 12:27:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:18.894456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ba9bd-7e85-419a-9f1f-4e0721940770 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:18.975075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:19.003180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175ebbe-0032-4044-92a4-e270765d4399 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:19.021486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ba9bd-7e85-419a-9f1f-4e0721940770 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:19.040780 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd455ac-b06b-4475-9cc3-a61a832e1e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:19.042430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd455ac-b06b-4475-9cc3-a61a832e1e75 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.042890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd455ac-b06b-4475-9cc3-a61a832e1e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.043964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd455ac-b06b-4475-9cc3-a61a832e1e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.056120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd455ac-b06b-4475-9cc3-a61a832e1e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:19.057904 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd455ac-b06b-4475-9cc3-a61a832e1e75 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:19.064347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2c0b2-8172-4a41-863a-b7058da4a128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:19.065109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2c0b2-8172-4a41-863a-b7058da4a128 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.065395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2c0b2-8172-4a41-863a-b7058da4a128 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.065826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2c0b2-8172-4a41-863a-b7058da4a128 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.067589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2c0b2-8172-4a41-863a-b7058da4a128 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:19.067909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2c0b2-8172-4a41-863a-b7058da4a128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:19.070301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2c0b2-8172-4a41-863a-b7058da4a128 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-2bd455ac-b06b-4475-9cc3-a61a832e1e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:19.079181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:19.080075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:19.080930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:19.080930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:19.080930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:19.080930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:19.087237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 377/747] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:27:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:19.087237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2c0b2-8172-4a41-863a-b7058da4a128 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:19.087577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 370/747] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:19.089387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd455ac-b06b-4475-9cc3-a61a832e1e75 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:19.152474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:19.209883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2c0b2-8172-4a41-863a-b7058da4a128 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:19.224283 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd455ac-b06b-4475-9cc3-a61a832e1e75 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:19.241888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:19.250103 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd6b56-064d-4974-b9e0-74216e0ebfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:19.250896 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd6b56-064d-4974-b9e0-74216e0ebfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.251175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd6b56-064d-4974-b9e0-74216e0ebfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.251448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd6b56-064d-4974-b9e0-74216e0ebfe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.259328 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b837392-ae23-4966-b435-8d9f433a22e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:19.260686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b837392-ae23-4966-b435-8d9f433a22e9 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.260973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b837392-ae23-4966-b435-8d9f433a22e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.261244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b837392-ae23-4966-b435-8d9f433a22e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.263223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd6b56-064d-4974-b9e0-74216e0ebfe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:19.264658 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd6b56-064d-4974-b9e0-74216e0ebfe3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:19.272167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b837392-ae23-4966-b435-8d9f433a22e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:19.273593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b837392-ae23-4966-b435-8d9f433a22e9 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-0bfd6b56-064d-4974-b9e0-74216e0ebfe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:19.283699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:19.284574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:19.285471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:19.285471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:19.285471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:19.285471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:19.289476 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 378/748] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:19.290107 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd6b56-064d-4974-b9e0-74216e0ebfe3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-2b837392-ae23-4966-b435-8d9f433a22e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:19.290695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:19.291673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:19.292560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:19.292560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:19.292560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:19.292560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:19.296899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 371/749] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:19.298269 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e7698-03ea-4842-8984-506587a12f4f None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.298559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e7698-03ea-4842-8984-506587a12f4f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.298839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e7698-03ea-4842-8984-506587a12f4f None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.301960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 372/750] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:19.301960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e7698-03ea-4842-8984-506587a12f4f None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:19.404060 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd6b56-064d-4974-b9e0-74216e0ebfe3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:19.415856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:19.421289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e7698-03ea-4842-8984-506587a12f4f None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:19.440464 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3b326-f165-431b-8ff3-0b5d1f5361b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:19.441129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3b326-f165-431b-8ff3-0b5d1f5361b8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.441394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3b326-f165-431b-8ff3-0b5d1f5361b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.441667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3b326-f165-431b-8ff3-0b5d1f5361b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.442441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3b326-f165-431b-8ff3-0b5d1f5361b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:19.442744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3b326-f165-431b-8ff3-0b5d1f5361b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:19.444109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3b326-f165-431b-8ff3-0b5d1f5361b8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:19.460241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 379/751] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:19.460241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3b326-f165-431b-8ff3-0b5d1f5361b8 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:19.462547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a24c6-78ee-451f-a8ef-8d127496ce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:19.463235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a24c6-78ee-451f-a8ef-8d127496ce4c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.463520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a24c6-78ee-451f-a8ef-8d127496ce4c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.463794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a24c6-78ee-451f-a8ef-8d127496ce4c None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.464525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a24c6-78ee-451f-a8ef-8d127496ce4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:19.464831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a24c6-78ee-451f-a8ef-8d127496ce4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:19.466249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a24c6-78ee-451f-a8ef-8d127496ce4c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:19.480313 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 373/752] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:19.481852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deef1e0-5dc3-4fd8-ae80-81b1bdf07aaa None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.482186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deef1e0-5dc3-4fd8-ae80-81b1bdf07aaa None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.482448 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deef1e0-5dc3-4fd8-ae80-81b1bdf07aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.501271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-7deef1e0-5dc3-4fd8-ae80-81b1bdf07aaa None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:19.502417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7deef1e0-5dc3-4fd8-ae80-81b1bdf07aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:19.506188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:19.565505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3b326-f165-431b-8ff3-0b5d1f5361b8 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:19.577953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 374/753] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:19.577953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deef1e0-5dc3-4fd8-ae80-81b1bdf07aaa None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:19.611483 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaba4ca-1f6a-4480-bff5-21f20cf2fc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:19.612144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaba4ca-1f6a-4480-bff5-21f20cf2fc9c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.612413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaba4ca-1f6a-4480-bff5-21f20cf2fc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.612688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaba4ca-1f6a-4480-bff5-21f20cf2fc9c None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.613404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaba4ca-1f6a-4480-bff5-21f20cf2fc9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:19.613706 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaba4ca-1f6a-4480-bff5-21f20cf2fc9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:19.615120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaba4ca-1f6a-4480-bff5-21f20cf2fc9c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:19.626956 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 380/754] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:19.627738 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaba4ca-1f6a-4480-bff5-21f20cf2fc9c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:19.675184 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:19.688310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deef1e0-5dc3-4fd8-ae80-81b1bdf07aaa None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:19.724886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6910-d6f1-4938-bbca-9e7cd12fec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:19.725838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6910-d6f1-4938-bbca-9e7cd12fec8a None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.726157 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6910-d6f1-4938-bbca-9e7cd12fec8a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.726543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6910-d6f1-4938-bbca-9e7cd12fec8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.733147 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaba4ca-1f6a-4480-bff5-21f20cf2fc9c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:19.734750 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:19.735358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec6910-d6f1-4938-bbca-9e7cd12fec8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:19.736820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec6910-d6f1-4938-bbca-9e7cd12fec8a None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-0dec6910-d6f1-4938-bbca-9e7cd12fec8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:19.752765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:19.753627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:19.755176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:19.755176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:19.755176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:19.755176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:19.755176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:19.755176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:19.755176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:19.755176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:19.755176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 375/755] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:19.755176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6910-d6f1-4938-bbca-9e7cd12fec8a None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:19.774576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7cdcc-c048-4997-a525-5fc3286e998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:19.775412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7cdcc-c048-4997-a525-5fc3286e998e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.776236 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7cdcc-c048-4997-a525-5fc3286e998e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.776560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7cdcc-c048-4997-a525-5fc3286e998e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.783653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7cdcc-c048-4997-a525-5fc3286e998e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:19.785102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7cdcc-c048-4997-a525-5fc3286e998e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-59d7cdcc-c048-4997-a525-5fc3286e998e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:19.800575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:19.801468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:19.803066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:19.803066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:19.803066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:19.803066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:19.803066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:19.803066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:19.803066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:19.803066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:19.803066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 381/756] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:19.803066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7cdcc-c048-4997-a525-5fc3286e998e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:19.865452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6910-d6f1-4938-bbca-9e7cd12fec8a None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:19.900371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309dbeb-448f-4788-a30e-234d194ba5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:19.901706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309dbeb-448f-4788-a30e-234d194ba5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.902014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309dbeb-448f-4788-a30e-234d194ba5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.902291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309dbeb-448f-4788-a30e-234d194ba5a6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.902939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309dbeb-448f-4788-a30e-234d194ba5a6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:19.904298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309dbeb-448f-4788-a30e-234d194ba5a6 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:19.909890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7cdcc-c048-4997-a525-5fc3286e998e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-3309dbeb-448f-4788-a30e-234d194ba5a6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:19.917286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:19.918279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:19.919103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 376/757] 10.222.0.60 () {56 vars in 1129 bytes} [Mon Jul 25 12:27:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:19.919934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309dbeb-448f-4788-a30e-234d194ba5a6 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:19.935617 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:19.963989 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c7c36-d456-4c24-a8ae-34683195084b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:19.964894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c7c36-d456-4c24-a8ae-34683195084b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:19.965366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c7c36-d456-4c24-a8ae-34683195084b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:19.965710 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c7c36-d456-4c24-a8ae-34683195084b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:19.974096 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c7c36-d456-4c24-a8ae-34683195084b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:19.975528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c7c36-d456-4c24-a8ae-34683195084b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-fd1c7c36-d456-4c24-a8ae-34683195084b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:19.992027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:19.992980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:19.993930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:19.993930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:19.993930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:19.993930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:19.993930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:19.993930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:19.993930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:19.993930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:19.993930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:27:19.993930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:19.993930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 382/758] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Jul 25 12:27:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:19.994549 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c7c36-d456-4c24-a8ae-34683195084b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:20.023799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:20.038982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309dbeb-448f-4788-a30e-234d194ba5a6 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:20.074894 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da632516-b29b-4d63-83dd-099f69ecb0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:20.075550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da632516-b29b-4d63-83dd-099f69ecb0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:20.075828 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da632516-b29b-4d63-83dd-099f69ecb0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:20.076099 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da632516-b29b-4d63-83dd-099f69ecb0d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:20.076882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da632516-b29b-4d63-83dd-099f69ecb0d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:20.077188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da632516-b29b-4d63-83dd-099f69ecb0d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:20.078633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da632516-b29b-4d63-83dd-099f69ecb0d3 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:20.097599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 377/759] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:20.097599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da632516-b29b-4d63-83dd-099f69ecb0d3 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:20.114646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c7c36-d456-4c24-a8ae-34683195084b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:20.148653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f8eca-9c92-4239-9519-db1e90eed859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:20.149313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f8eca-9c92-4239-9519-db1e90eed859 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:20.149596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f8eca-9c92-4239-9519-db1e90eed859 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:20.149873 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f8eca-9c92-4239-9519-db1e90eed859 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:20.151246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f8eca-9c92-4239-9519-db1e90eed859 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:20.151558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f8eca-9c92-4239-9519-db1e90eed859 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:20.153166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f8eca-9c92-4239-9519-db1e90eed859 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:20.170034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 383/760] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:20.170034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f8eca-9c92-4239-9519-db1e90eed859 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:20.185170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:20.221660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da632516-b29b-4d63-83dd-099f69ecb0d3 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:20.259532 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:20.259720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60e51b-0bce-4aa4-9a54-8012a1d96570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:20.260336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60e51b-0bce-4aa4-9a54-8012a1d96570 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:20.260691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60e51b-0bce-4aa4-9a54-8012a1d96570 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:20.260910 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60e51b-0bce-4aa4-9a54-8012a1d96570 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:20.261773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60e51b-0bce-4aa4-9a54-8012a1d96570 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:20.261879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60e51b-0bce-4aa4-9a54-8012a1d96570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:20.263407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60e51b-0bce-4aa4-9a54-8012a1d96570 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:20.268113 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 378/761] 10.222.0.60 () {56 vars in 1124 bytes} [Mon Jul 25 12:27:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:20.273320 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60e51b-0bce-4aa4-9a54-8012a1d96570 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:20.279666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f8eca-9c92-4239-9519-db1e90eed859 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:20.315698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696cb12-8190-4e77-8084-ab09a06dabae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:20.316363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696cb12-8190-4e77-8084-ab09a06dabae None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:20.316643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696cb12-8190-4e77-8084-ab09a06dabae None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:20.316926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696cb12-8190-4e77-8084-ab09a06dabae None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:20.317749 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696cb12-8190-4e77-8084-ab09a06dabae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:20.318085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696cb12-8190-4e77-8084-ab09a06dabae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:20.319502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696cb12-8190-4e77-8084-ab09a06dabae None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:20.335828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 384/762] 10.222.0.60 () {56 vars in 1149 bytes} [Mon Jul 25 12:27:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:20.336646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696cb12-8190-4e77-8084-ab09a06dabae None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:20.376169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60e51b-0bce-4aa4-9a54-8012a1d96570 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:20.410246 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6fed1-fc13-47e2-bfe2-78adafd2cc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:20.410896 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6fed1-fc13-47e2-bfe2-78adafd2cc21 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:20.411177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6fed1-fc13-47e2-bfe2-78adafd2cc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:20.411449 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6fed1-fc13-47e2-bfe2-78adafd2cc21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:20.412272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6fed1-fc13-47e2-bfe2-78adafd2cc21 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=e56b0d48e08c4f778a7c509c10edc871) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:20.412580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6fed1-fc13-47e2-bfe2-78adafd2cc21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:20.414025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6fed1-fc13-47e2-bfe2-78adafd2cc21 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:20.417406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:20.424920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 379/763] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:20 2022] GET /identity/v3/role_assignments?user.id=e56b0d48e08c4f778a7c509c10edc871&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:20.425845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6fed1-fc13-47e2-bfe2-78adafd2cc21 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:20.438402 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696cb12-8190-4e77-8084-ab09a06dabae None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:20.478338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fec619-f303-4454-90c9-ee7e2ed2e9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:20.479780 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fec619-f303-4454-90c9-ee7e2ed2e9ad None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:20.480055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fec619-f303-4454-90c9-ee7e2ed2e9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:20.480327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fec619-f303-4454-90c9-ee7e2ed2e9ad None admin] PATH_INFO: `/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/ccf2dedaf7a449d2b51b9e9f3b420091/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:20.482720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fec619-f303-4454-90c9-ee7e2ed2e9ad None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff5a41f8ca6438295c512792d2022e4, user_id=ccf2dedaf7a449d2b51b9e9f3b420091, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:20.484199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fec619-f303-4454-90c9-ee7e2ed2e9ad None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:20.495583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:20.506218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 385/764] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:27:20 2022] PUT /identity/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/ccf2dedaf7a449d2b51b9e9f3b420091/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:27:20.507796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77c7da-daa4-43b0-b491-ca4b61988b4b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:20.508083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77c7da-daa4-43b0-b491-ca4b61988b4b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:20.508358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77c7da-daa4-43b0-b491-ca4b61988b4b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:20.514729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 386/765] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:20.515743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77c7da-daa4-43b0-b491-ca4b61988b4b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:20.556212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6fed1-fc13-47e2-bfe2-78adafd2cc21 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:20.609707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92f7a5-cc04-4897-842d-9d806760496f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:20.610423 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92f7a5-cc04-4897-842d-9d806760496f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:20.610837 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92f7a5-cc04-4897-842d-9d806760496f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:20.611185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92f7a5-cc04-4897-842d-9d806760496f None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:20.611988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92f7a5-cc04-4897-842d-9d806760496f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:20.612365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92f7a5-cc04-4897-842d-9d806760496f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:20.613835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92f7a5-cc04-4897-842d-9d806760496f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:20.620004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 380/766] 10.222.0.60 () {56 vars in 1124 bytes} [Mon Jul 25 12:27:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:20.621457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b10fce-69c4-4cce-9f81-b24f83140c1a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:20.621816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b10fce-69c4-4cce-9f81-b24f83140c1a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:20.622208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b10fce-69c4-4cce-9f81-b24f83140c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:20.641530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-e7b10fce-69c4-4cce-9f81-b24f83140c1a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:20.642940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e7b10fce-69c4-4cce-9f81-b24f83140c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:20.667578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77c7da-daa4-43b0-b491-ca4b61988b4b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:20.708242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab4c01-8c36-4f20-9f8e-3c2e95818b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:20.709396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab4c01-8c36-4f20-9f8e-3c2e95818b9d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:20.709905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab4c01-8c36-4f20-9f8e-3c2e95818b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:20.710420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab4c01-8c36-4f20-9f8e-3c2e95818b9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:20.712100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:20.725950 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 381/767] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:20.725950 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b10fce-69c4-4cce-9f81-b24f83140c1a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:20.785687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:20.788040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab4c01-8c36-4f20-9f8e-3c2e95818b9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:20.789496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab4c01-8c36-4f20-9f8e-3c2e95818b9d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:20.848836 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b10fce-69c4-4cce-9f81-b24f83140c1a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:20.886501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eeaaca-dd8c-4dc0-92a5-0a8e59fc6a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:20.887563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eeaaca-dd8c-4dc0-92a5-0a8e59fc6a00 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:20.888063 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eeaaca-dd8c-4dc0-92a5-0a8e59fc6a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:20.888546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eeaaca-dd8c-4dc0-92a5-0a8e59fc6a00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:20.889735 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eeaaca-dd8c-4dc0-92a5-0a8e59fc6a00 None admin] RBAC: Adding query filter params (interface=public, service_id=685bef231476402691624317d563a230, region_id=RegionOne) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:20.890330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eeaaca-dd8c-4dc0-92a5-0a8e59fc6a00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:20.891955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eeaaca-dd8c-4dc0-92a5-0a8e59fc6a00 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:20.893515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 387/768] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:27:20.894696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab4c01-8c36-4f20-9f8e-3c2e95818b9d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:20.906577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 382/769] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Jul 25 12:27:20 2022] GET /identity/v3/endpoints?service_id=685bef231476402691624317d563a230&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:20.907809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eeaaca-dd8c-4dc0-92a5-0a8e59fc6a00 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:20.982360 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:20.997669 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:21.000964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab4c01-8c36-4f20-9f8e-3c2e95818b9d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:21.024219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eeaaca-dd8c-4dc0-92a5-0a8e59fc6a00 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:21.037932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed6316-d3be-4a8b-beb8-10b6fa31bbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:21.038809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed6316-d3be-4a8b-beb8-10b6fa31bbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:21.039155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed6316-d3be-4a8b-beb8-10b6fa31bbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:21.039496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed6316-d3be-4a8b-beb8-10b6fa31bbb7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:21.040428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed6316-d3be-4a8b-beb8-10b6fa31bbb7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:21.042285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed6316-d3be-4a8b-beb8-10b6fa31bbb7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-28ed6316-d3be-4a8b-beb8-10b6fa31bbb7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:21.054442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:21.055443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:27:21.056263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:21.061205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 388/770] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:20 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:21.062200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed6316-d3be-4a8b-beb8-10b6fa31bbb7 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:21.066393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb90fb4-e9c8-40a1-b61f-4ebb4e65839e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:21.067413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb90fb4-e9c8-40a1-b61f-4ebb4e65839e None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:21.067881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb90fb4-e9c8-40a1-b61f-4ebb4e65839e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:21.068333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb90fb4-e9c8-40a1-b61f-4ebb4e65839e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:21.080714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb90fb4-e9c8-40a1-b61f-4ebb4e65839e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:21.082712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb90fb4-e9c8-40a1-b61f-4ebb4e65839e None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-abb90fb4-e9c8-40a1-b61f-4ebb4e65839e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:21.101576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:21.102695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:21.104110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:21.104110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:21.104110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:21.104110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:21.106892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 383/771] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:21.108049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb90fb4-e9c8-40a1-b61f-4ebb4e65839e None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:21.165208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed6316-d3be-4a8b-beb8-10b6fa31bbb7 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:21.203959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b5aaa-cbfe-4a5d-834e-6dbd87ab1c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:21.204752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b5aaa-cbfe-4a5d-834e-6dbd87ab1c41 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:21.205062 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b5aaa-cbfe-4a5d-834e-6dbd87ab1c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:21.205406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b5aaa-cbfe-4a5d-834e-6dbd87ab1c41 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:21.206458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b5aaa-cbfe-4a5d-834e-6dbd87ab1c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:21.207754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b5aaa-cbfe-4a5d-834e-6dbd87ab1c41 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:21.222461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 389/772] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Jul 25 12:27:21 2022] GET /identity/v3/services => generated 772 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:21.223724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b5aaa-cbfe-4a5d-834e-6dbd87ab1c41 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:21.226906 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb90fb4-e9c8-40a1-b61f-4ebb4e65839e None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:21.242339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:21.270932 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598226bf-3101-4e15-8ca4-1fdb96ace38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:21.272034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598226bf-3101-4e15-8ca4-1fdb96ace38b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:21.272634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:21.273467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598226bf-3101-4e15-8ca4-1fdb96ace38b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:21.274082 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598226bf-3101-4e15-8ca4-1fdb96ace38b None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:21.275086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598226bf-3101-4e15-8ca4-1fdb96ace38b None admin] RBAC: Adding query filter params (name=network) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:21.276198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598226bf-3101-4e15-8ca4-1fdb96ace38b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:21.277832 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598226bf-3101-4e15-8ca4-1fdb96ace38b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:21.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 384/773] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:21.294837 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598226bf-3101-4e15-8ca4-1fdb96ace38b None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:21.364076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b5aaa-cbfe-4a5d-834e-6dbd87ab1c41 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:21.401336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce9ee3-0cec-4335-a4af-52a137169730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:21.402062 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce9ee3-0cec-4335-a4af-52a137169730 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:21.402345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce9ee3-0cec-4335-a4af-52a137169730 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:21.402612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce9ee3-0cec-4335-a4af-52a137169730 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:21.403787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce9ee3-0cec-4335-a4af-52a137169730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:21.403787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce9ee3-0cec-4335-a4af-52a137169730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:21.405114 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce9ee3-0cec-4335-a4af-52a137169730 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:21.421323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 390/774] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:21.425046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce9ee3-0cec-4335-a4af-52a137169730 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:21.496846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598226bf-3101-4e15-8ca4-1fdb96ace38b None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:21.524384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:21.538588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdce56f-36db-4882-8c34-f0cab412381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:21.540388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdce56f-36db-4882-8c34-f0cab412381f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:21.540906 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdce56f-36db-4882-8c34-f0cab412381f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:21.542015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdce56f-36db-4882-8c34-f0cab412381f None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:21.542259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce9ee3-0cec-4335-a4af-52a137169730 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:21.543166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdce56f-36db-4882-8c34-f0cab412381f None admin] RBAC: Adding query filter params (type=network) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:21.543778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdce56f-36db-4882-8c34-f0cab412381f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:21.545481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdce56f-36db-4882-8c34-f0cab412381f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:21.552200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 385/775] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:21.557628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdce56f-36db-4882-8c34-f0cab412381f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:21.585617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fac7b-a6df-4668-a2ac-d31473703a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:21.586769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fac7b-a6df-4668-a2ac-d31473703a7e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:21.587270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fac7b-a6df-4668-a2ac-d31473703a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:21.587759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fac7b-a6df-4668-a2ac-d31473703a7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:21.596351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fac7b-a6df-4668-a2ac-d31473703a7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:21.598175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fac7b-a6df-4668-a2ac-d31473703a7e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:21.618332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-779fac7b-a6df-4668-a2ac-d31473703a7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:21.629941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:21.630878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:21.631783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:21.631783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:21.631783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:21.631783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:21.631783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:21.631783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:21.631783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:21.631783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:21.631783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 391/776] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:21.638914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fac7b-a6df-4668-a2ac-d31473703a7e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:21.838769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fac7b-a6df-4668-a2ac-d31473703a7e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:21.874297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788b999-6300-41ce-8686-a1e39619fabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:21.874878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788b999-6300-41ce-8686-a1e39619fabd None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:21.875160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788b999-6300-41ce-8686-a1e39619fabd None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:21.875436 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788b999-6300-41ce-8686-a1e39619fabd None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:21.876222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788b999-6300-41ce-8686-a1e39619fabd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:21.876529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788b999-6300-41ce-8686-a1e39619fabd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:21.877970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788b999-6300-41ce-8686-a1e39619fabd None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:21.880724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdce56f-36db-4882-8c34-f0cab412381f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:21.894564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 392/777] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:21.902875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788b999-6300-41ce-8686-a1e39619fabd None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:21.921070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:21.947039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67d4d6-91a0-482c-b68b-50203e6328b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:21.948234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67d4d6-91a0-482c-b68b-50203e6328b8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:21.948567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67d4d6-91a0-482c-b68b-50203e6328b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:21.948841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67d4d6-91a0-482c-b68b-50203e6328b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:21.950331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67d4d6-91a0-482c-b68b-50203e6328b8 None admin] RBAC: Adding query filter params (interface=public, service_id=426aa8bafa3843ed93147631f4fe23f4, region_id=RegionOne) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:21.950645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67d4d6-91a0-482c-b68b-50203e6328b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:21.952280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67d4d6-91a0-482c-b68b-50203e6328b8 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:21.962118 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 386/778] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Jul 25 12:27:21 2022] GET /identity/v3/endpoints?service_id=426aa8bafa3843ed93147631f4fe23f4&interface=public®ion_id=RegionOne => generated 194 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:21.969834 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67d4d6-91a0-482c-b68b-50203e6328b8 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:22.081754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788b999-6300-41ce-8686-a1e39619fabd None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:22.093994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67d4d6-91a0-482c-b68b-50203e6328b8 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:22.107082 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:22.118525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d809c-89aa-4545-a513-795f893cb2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:22.119742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d809c-89aa-4545-a513-795f893cb2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:22.120027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d809c-89aa-4545-a513-795f893cb2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:22.120298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d809c-89aa-4545-a513-795f893cb2f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:22.121127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d809c-89aa-4545-a513-795f893cb2f2 None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=6428e810f10c434ba32d073a542b1de7) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:22.121437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d809c-89aa-4545-a513-795f893cb2f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:22.123250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d809c-89aa-4545-a513-795f893cb2f2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:22.130101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 393/779] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:21 2022] GET /identity/v3/role_assignments?user.id=6428e810f10c434ba32d073a542b1de7&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:22.132031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfcb12-71a0-4868-9489-7380a84fe4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:22.132712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfcb12-71a0-4868-9489-7380a84fe4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:22.132998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfcb12-71a0-4868-9489-7380a84fe4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:22.133280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfcb12-71a0-4868-9489-7380a84fe4c8 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:22.134011 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cfcb12-71a0-4868-9489-7380a84fe4c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:22.135996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cfcb12-71a0-4868-9489-7380a84fe4c8 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:22.142638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 387/780] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Jul 25 12:27:21 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:22.643723 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496638a5-6a1f-44ea-aa57-65e31f8bd938 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:22.644261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496638a5-6a1f-44ea-aa57-65e31f8bd938 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:22.644261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496638a5-6a1f-44ea-aa57-65e31f8bd938 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:22.645955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 394/781] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:22.651319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ab002-eb5a-4d08-a0ed-842ac2884b28 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:22.651762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ab002-eb5a-4d08-a0ed-842ac2884b28 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:22.651978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ab002-eb5a-4d08-a0ed-842ac2884b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:22.683247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-903ab002-eb5a-4d08-a0ed-842ac2884b28 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:22.684481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-903ab002-eb5a-4d08-a0ed-842ac2884b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:22.771413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 388/782] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:22.806008 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496638a5-6a1f-44ea-aa57-65e31f8bd938 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:22.898141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e045e5c-b0ff-4be9-9812-0dee234da385 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:22.898141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e045e5c-b0ff-4be9-9812-0dee234da385 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:22.898141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e045e5c-b0ff-4be9-9812-0dee234da385 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:22.899743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 389/783] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:22.908522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:22.912727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a9fa2-1e14-4ec1-8eba-5f68002f0f21 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:22.913017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a9fa2-1e14-4ec1-8eba-5f68002f0f21 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:22.913298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a9fa2-1e14-4ec1-8eba-5f68002f0f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:22.946447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-fe2a9fa2-1e14-4ec1-8eba-5f68002f0f21 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:22.947666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fe2a9fa2-1e14-4ec1-8eba-5f68002f0f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:22.963633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496638a5-6a1f-44ea-aa57-65e31f8bd938 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:23.044214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 390/784] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:23.046626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c9c39-2083-455f-b2eb-f8ebad933ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:23.047487 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c9c39-2083-455f-b2eb-f8ebad933ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:23.047769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c9c39-2083-455f-b2eb-f8ebad933ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:23.048041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c9c39-2083-455f-b2eb-f8ebad933ee4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:23.059276 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c9c39-2083-455f-b2eb-f8ebad933ee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-371c9c39-2083-455f-b2eb-f8ebad933ee4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:23.067117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:23.068026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:23.068846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 395/785] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:23.077986 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a9fa2-1e14-4ec1-8eba-5f68002f0f21 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:23.082145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c9c39-2083-455f-b2eb-f8ebad933ee4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:23.120805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:23.211180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c9c39-2083-455f-b2eb-f8ebad933ee4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:23.242283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:23.248113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46497a68-93da-4590-a906-d3bde6f96b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:23.248961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46497a68-93da-4590-a906-d3bde6f96b1e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:23.249248 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46497a68-93da-4590-a906-d3bde6f96b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:23.249524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46497a68-93da-4590-a906-d3bde6f96b1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:23.260677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46497a68-93da-4590-a906-d3bde6f96b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-46497a68-93da-4590-a906-d3bde6f96b1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:23.274051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:23.274963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:23.276371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 396/786] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:23.279940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46497a68-93da-4590-a906-d3bde6f96b1e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:23.307664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a9fa2-1e14-4ec1-8eba-5f68002f0f21 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:23.372141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28a333-5890-48a4-b5f6-afddc3273c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:23.372809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28a333-5890-48a4-b5f6-afddc3273c93 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:23.373086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28a333-5890-48a4-b5f6-afddc3273c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:23.373358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28a333-5890-48a4-b5f6-afddc3273c93 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:23.374243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28a333-5890-48a4-b5f6-afddc3273c93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:23.374550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28a333-5890-48a4-b5f6-afddc3273c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:23.375909 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28a333-5890-48a4-b5f6-afddc3273c93 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:23.393562 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 391/787] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:23.401059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28a333-5890-48a4-b5f6-afddc3273c93 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:23.473814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46497a68-93da-4590-a906-d3bde6f96b1e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:23.518255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:23.531594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28a333-5890-48a4-b5f6-afddc3273c93 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:23.553847 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db8601-c3be-48d6-b7c9-9e66113b71fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:23.554559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db8601-c3be-48d6-b7c9-9e66113b71fc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:23.554893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db8601-c3be-48d6-b7c9-9e66113b71fc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:23.555176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db8601-c3be-48d6-b7c9-9e66113b71fc None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:23.555932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db8601-c3be-48d6-b7c9-9e66113b71fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:23.556238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db8601-c3be-48d6-b7c9-9e66113b71fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:23.558191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db8601-c3be-48d6-b7c9-9e66113b71fc None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:23.574450 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 397/788] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:23.582541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db8601-c3be-48d6-b7c9-9e66113b71fc None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:23.589460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdcd64-7e02-4050-9698-8b8629ff4e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:23.590279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdcd64-7e02-4050-9698-8b8629ff4e8a None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:23.590814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdcd64-7e02-4050-9698-8b8629ff4e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:23.591088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdcd64-7e02-4050-9698-8b8629ff4e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:23.612620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdcd64-7e02-4050-9698-8b8629ff4e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:23.614097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdcd64-7e02-4050-9698-8b8629ff4e8a None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-b4bdcd64-7e02-4050-9698-8b8629ff4e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:23.654178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:23.655539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:23.656425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:23.656425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:23.656425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:23.656425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:23.656425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 392/789] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:23.661381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdcd64-7e02-4050-9698-8b8629ff4e8a None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:23.666029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:23.749707 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db8601-c3be-48d6-b7c9-9e66113b71fc None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:23.814018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdcd64-7e02-4050-9698-8b8629ff4e8a None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:23.815778 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a8e33-6a24-4469-b63c-cefbc2176b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:23.816669 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a8e33-6a24-4469-b63c-cefbc2176b6b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:23.816955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a8e33-6a24-4469-b63c-cefbc2176b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:23.817231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a8e33-6a24-4469-b63c-cefbc2176b6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:23.828726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a8e33-6a24-4469-b63c-cefbc2176b6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:23.830455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a8e33-6a24-4469-b63c-cefbc2176b6b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-f55a8e33-6a24-4469-b63c-cefbc2176b6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:23.848131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:23.849097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:23.850029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:23.850029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:23.850029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:23.850029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:23.850029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 398/790] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:23.851577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deead59c-62c2-4415-8f9c-3210bcc9f43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:23.851731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8487a81-f8e0-4c6d-b728-d04348ddb608 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:23.851731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8487a81-f8e0-4c6d-b728-d04348ddb608 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:23.851731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8487a81-f8e0-4c6d-b728-d04348ddb608 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:23.851919 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deead59c-62c2-4415-8f9c-3210bcc9f43c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:23.851919 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deead59c-62c2-4415-8f9c-3210bcc9f43c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:23.852176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deead59c-62c2-4415-8f9c-3210bcc9f43c None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:23.853982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deead59c-62c2-4415-8f9c-3210bcc9f43c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:23.854094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 399/791] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:23.854169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deead59c-62c2-4415-8f9c-3210bcc9f43c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:23.854737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8487a81-f8e0-4c6d-b728-d04348ddb608 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:23.854923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deead59c-62c2-4415-8f9c-3210bcc9f43c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:23.864766 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:23.869620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 393/792] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:23.870936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9c334-e183-4a58-b737-2b1ca7958dd8 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:23.871278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9c334-e183-4a58-b737-2b1ca7958dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:23.871552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9c334-e183-4a58-b737-2b1ca7958dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:23.894100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-97d9c334-e183-4a58-b737-2b1ca7958dd8 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:23.895446 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-97d9c334-e183-4a58-b737-2b1ca7958dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:23.977685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 394/793] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:23.979462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9c334-e183-4a58-b737-2b1ca7958dd8 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:23.986857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8487a81-f8e0-4c6d-b728-d04348ddb608 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:24.010522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:24.039198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dae569-6621-4a83-a8a7-62dc0cf4c864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:24.039959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dae569-6621-4a83-a8a7-62dc0cf4c864 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.040245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dae569-6621-4a83-a8a7-62dc0cf4c864 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.040519 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dae569-6621-4a83-a8a7-62dc0cf4c864 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.041788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dae569-6621-4a83-a8a7-62dc0cf4c864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:24.042118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dae569-6621-4a83-a8a7-62dc0cf4c864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:24.043726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dae569-6621-4a83-a8a7-62dc0cf4c864 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:24.056238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 400/794] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:24.057797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dae569-6621-4a83-a8a7-62dc0cf4c864 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:24.100293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9c334-e183-4a58-b737-2b1ca7958dd8 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:24.131837 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:24.139276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce81ac1-1062-4eda-8630-2053dd3eaff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:24.140169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce81ac1-1062-4eda-8630-2053dd3eaff3 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.140514 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce81ac1-1062-4eda-8630-2053dd3eaff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.140923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce81ac1-1062-4eda-8630-2053dd3eaff3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.149526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce81ac1-1062-4eda-8630-2053dd3eaff3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:24.151334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce81ac1-1062-4eda-8630-2053dd3eaff3 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:24.167045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dae569-6621-4a83-a8a7-62dc0cf4c864 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-cce81ac1-1062-4eda-8630-2053dd3eaff3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:24.172678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:24.174845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:24.177068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:24.177068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:24.177068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:24.177068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:24.177068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:24.177068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:24.177068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:24.177068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:24.177068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:27:24.177068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:24.177068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 395/795] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:24.177068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce81ac1-1062-4eda-8630-2053dd3eaff3 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:24.209015 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bb97c-ac0d-48b9-9a85-51641fa11991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:24.209988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bb97c-ac0d-48b9-9a85-51641fa11991 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.210231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bb97c-ac0d-48b9-9a85-51641fa11991 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.210506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bb97c-ac0d-48b9-9a85-51641fa11991 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.211161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bb97c-ac0d-48b9-9a85-51641fa11991 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:24.212623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bb97c-ac0d-48b9-9a85-51641fa11991 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-695bb97c-ac0d-48b9-9a85-51641fa11991 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:24.225455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:24.226606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:24.227543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 401/796] 10.222.0.60 () {56 vars in 1129 bytes} [Mon Jul 25 12:27:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:24.228378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bb97c-ac0d-48b9-9a85-51641fa11991 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:24.261377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:24.302636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce81ac1-1062-4eda-8630-2053dd3eaff3 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:24.338387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bb97c-ac0d-48b9-9a85-51641fa11991 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:24.339009 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b945e8-f064-41f0-ac9a-373ef23306c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:24.340114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b945e8-f064-41f0-ac9a-373ef23306c7 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.340731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b945e8-f064-41f0-ac9a-373ef23306c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.341159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b945e8-f064-41f0-ac9a-373ef23306c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.349847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b945e8-f064-41f0-ac9a-373ef23306c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:24.351588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b945e8-f064-41f0-ac9a-373ef23306c7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-c2b945e8-f064-41f0-ac9a-373ef23306c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:24.368255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:24.369472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:24.370517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:24.370517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:24.370517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:24.370517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:24.370517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:24.370517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:24.370517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:24.370517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:24.370517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:24.370517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:24.372658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 396/797] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Jul 25 12:27:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:24.373563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b945e8-f064-41f0-ac9a-373ef23306c7 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:24.374927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c0a38-d6e3-4bd0-8eae-dbe45f8e9e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:24.375594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c0a38-d6e3-4bd0-8eae-dbe45f8e9e56 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.375865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c0a38-d6e3-4bd0-8eae-dbe45f8e9e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.376142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c0a38-d6e3-4bd0-8eae-dbe45f8e9e56 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.377290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c0a38-d6e3-4bd0-8eae-dbe45f8e9e56 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:24.377521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c0a38-d6e3-4bd0-8eae-dbe45f8e9e56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:24.379152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c0a38-d6e3-4bd0-8eae-dbe45f8e9e56 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:24.402407 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 402/798] 10.222.0.60 () {56 vars in 1135 bytes} [Mon Jul 25 12:27:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:24.403388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c0a38-d6e3-4bd0-8eae-dbe45f8e9e56 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:24.429945 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:24.485764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b945e8-f064-41f0-ac9a-373ef23306c7 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:24.505074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:24.518929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c0a38-d6e3-4bd0-8eae-dbe45f8e9e56 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:24.523373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e67e39-0fc4-49ae-8e89-b2b63363178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:24.524036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e67e39-0fc4-49ae-8e89-b2b63363178d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.524314 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e67e39-0fc4-49ae-8e89-b2b63363178d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.524585 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e67e39-0fc4-49ae-8e89-b2b63363178d None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.525808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e67e39-0fc4-49ae-8e89-b2b63363178d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:24.526175 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e67e39-0fc4-49ae-8e89-b2b63363178d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:24.527540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e67e39-0fc4-49ae-8e89-b2b63363178d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:24.532600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 397/799] 10.222.0.60 () {56 vars in 1124 bytes} [Mon Jul 25 12:27:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:24.533366 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e67e39-0fc4-49ae-8e89-b2b63363178d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:24.560397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2f85f-09a2-4f5f-9b59-b2f35ebd25c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:24.561169 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2f85f-09a2-4f5f-9b59-b2f35ebd25c8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.561751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2f85f-09a2-4f5f-9b59-b2f35ebd25c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.561877 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2f85f-09a2-4f5f-9b59-b2f35ebd25c8 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.562660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2f85f-09a2-4f5f-9b59-b2f35ebd25c8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:24.562970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2f85f-09a2-4f5f-9b59-b2f35ebd25c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:24.564399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2f85f-09a2-4f5f-9b59-b2f35ebd25c8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:24.586765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 403/800] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Jul 25 12:27:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:24.586765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7e84c-9079-478e-a42c-fc59c2740c46 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.586765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7e84c-9079-478e-a42c-fc59c2740c46 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.586765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7e84c-9079-478e-a42c-fc59c2740c46 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.586765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 404/801] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:24.586765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7e84c-9079-478e-a42c-fc59c2740c46 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:24.646540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e67e39-0fc4-49ae-8e89-b2b63363178d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:24.660891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:24.681061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d019ed1-61f7-47f3-bbf8-5ba138526d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:24.681870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d019ed1-61f7-47f3-bbf8-5ba138526d8a None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.682183 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d019ed1-61f7-47f3-bbf8-5ba138526d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.682460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d019ed1-61f7-47f3-bbf8-5ba138526d8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.691875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7e84c-9079-478e-a42c-fc59c2740c46 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:24.692773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d019ed1-61f7-47f3-bbf8-5ba138526d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:24.694247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d019ed1-61f7-47f3-bbf8-5ba138526d8a None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-8d019ed1-61f7-47f3-bbf8-5ba138526d8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:24.711306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:24.712179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:24.713626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:24.713626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:24.713626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:24.713626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:24.713626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 398/802] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:24.713983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d019ed1-61f7-47f3-bbf8-5ba138526d8a None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:24.730776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cf47f-6c1c-4523-8ed7-9af1fd43359e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:24.731699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cf47f-6c1c-4523-8ed7-9af1fd43359e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.731982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cf47f-6c1c-4523-8ed7-9af1fd43359e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.732252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cf47f-6c1c-4523-8ed7-9af1fd43359e None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.732953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cf47f-6c1c-4523-8ed7-9af1fd43359e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:24.733255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cf47f-6c1c-4523-8ed7-9af1fd43359e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:24.734636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cf47f-6c1c-4523-8ed7-9af1fd43359e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:24.739358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 405/803] 10.222.0.60 () {56 vars in 1124 bytes} [Mon Jul 25 12:27:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:24.740491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bda9fb-d7ca-403e-a3b2-520a750ad2c2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.740784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bda9fb-d7ca-403e-a3b2-520a750ad2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.741059 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bda9fb-d7ca-403e-a3b2-520a750ad2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.758028 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-43bda9fb-d7ca-403e-a3b2-520a750ad2c2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:24.759201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-43bda9fb-d7ca-403e-a3b2-520a750ad2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:24.762553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:24.811830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d019ed1-61f7-47f3-bbf8-5ba138526d8a None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:24.827290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 406/804] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:24.828575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af9264-9ac1-4558-8ed3-bda917faa916 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.828860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af9264-9ac1-4558-8ed3-bda917faa916 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.829132 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af9264-9ac1-4558-8ed3-bda917faa916 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.830405 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 407/805] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:24.831079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af9264-9ac1-4558-8ed3-bda917faa916 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:24.855058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30361fe-9670-4725-a2ef-135211140245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:24.855700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30361fe-9670-4725-a2ef-135211140245 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30361fe-9670-4725-a2ef-135211140245 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.856513 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30361fe-9670-4725-a2ef-135211140245 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.888709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:24.923167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30361fe-9670-4725-a2ef-135211140245 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:24.924804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30361fe-9670-4725-a2ef-135211140245 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:24.939090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af9264-9ac1-4558-8ed3-bda917faa916 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:24.973643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd6108-d1b2-4e25-aff1-d8c4980291d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:24.974338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd6108-d1b2-4e25-aff1-d8c4980291d1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:24.974621 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd6108-d1b2-4e25-aff1-d8c4980291d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:24.974898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd6108-d1b2-4e25-aff1-d8c4980291d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:24.975619 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd6108-d1b2-4e25-aff1-d8c4980291d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:24.975926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd6108-d1b2-4e25-aff1-d8c4980291d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:24.977285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd6108-d1b2-4e25-aff1-d8c4980291d1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:24.985597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:24.993539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 408/806] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:24.993539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd6108-d1b2-4e25-aff1-d8c4980291d1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:25.024268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 399/807] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:27:25.025793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f335bff-7be3-4267-8359-d021c5aed8e1 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:25.026106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f335bff-7be3-4267-8359-d021c5aed8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:25.026380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f335bff-7be3-4267-8359-d021c5aed8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:25.043095 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-8f335bff-7be3-4267-8359-d021c5aed8e1 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:25.044293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8f335bff-7be3-4267-8359-d021c5aed8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:25.092615 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd6108-d1b2-4e25-aff1-d8c4980291d1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:25.101258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:25.109897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 400/808] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:25.110715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f335bff-7be3-4267-8359-d021c5aed8e1 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:25.127860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1b8b4-7d5d-4a1c-84cf-f6ec1058ec4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:25.128584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1b8b4-7d5d-4a1c-84cf-f6ec1058ec4b None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:25.128782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1b8b4-7d5d-4a1c-84cf-f6ec1058ec4b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:25.129052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1b8b4-7d5d-4a1c-84cf-f6ec1058ec4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:25.129788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1b8b4-7d5d-4a1c-84cf-f6ec1058ec4b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:25.131200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1b8b4-7d5d-4a1c-84cf-f6ec1058ec4b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:25.152887 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 409/809] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:27:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:27:25.153688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1b8b4-7d5d-4a1c-84cf-f6ec1058ec4b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:25.223309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f335bff-7be3-4267-8359-d021c5aed8e1 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:25.225965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:25.271709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff088e-d960-42c8-9f5f-9c453b13d548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:25.272613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff088e-d960-42c8-9f5f-9c453b13d548 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:25.272969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff088e-d960-42c8-9f5f-9c453b13d548 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:25.273348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff088e-d960-42c8-9f5f-9c453b13d548 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:25.279426 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1b8b4-7d5d-4a1c-84cf-f6ec1058ec4b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:25.285131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff088e-d960-42c8-9f5f-9c453b13d548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-b9ff088e-d960-42c8-9f5f-9c453b13d548 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:25.294421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:25.295634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 401/810] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:25.301527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff088e-d960-42c8-9f5f-9c453b13d548 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:25.329322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc519d-cc25-4e48-8fcd-4b8a5c8c44b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:25.330330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc519d-cc25-4e48-8fcd-4b8a5c8c44b0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:25.330604 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc519d-cc25-4e48-8fcd-4b8a5c8c44b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:25.330882 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc519d-cc25-4e48-8fcd-4b8a5c8c44b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:25.339219 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc519d-cc25-4e48-8fcd-4b8a5c8c44b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:25.340667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc519d-cc25-4e48-8fcd-4b8a5c8c44b0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c0dc519d-cc25-4e48-8fcd-4b8a5c8c44b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:25.358410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:25.359495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:25.360477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:25.360477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:25.360477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:25.360477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:25.360477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:25.360477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:25.360477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:25.360477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:25.360477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 410/811] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:25.361024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc519d-cc25-4e48-8fcd-4b8a5c8c44b0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:25.362028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:25.413586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff088e-d960-42c8-9f5f-9c453b13d548 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:25.450022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa74a95-950a-461b-8a69-e82a1a3e6b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:25.450807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa74a95-950a-461b-8a69-e82a1a3e6b96 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:25.451088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa74a95-950a-461b-8a69-e82a1a3e6b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:25.451358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa74a95-950a-461b-8a69-e82a1a3e6b96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:25.462329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa74a95-950a-461b-8a69-e82a1a3e6b96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:25.463809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa74a95-950a-461b-8a69-e82a1a3e6b96 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:25.467962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc519d-cc25-4e48-8fcd-4b8a5c8c44b0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-6aa74a95-950a-461b-8a69-e82a1a3e6b96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:25.482624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:25.483715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:25.484632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:25.484632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:25.484632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:25.484632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:25.484632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 402/812] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:25.484632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa74a95-950a-461b-8a69-e82a1a3e6b96 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:25.493290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:25.514422 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3c82b-7498-4167-add3-d0093d83f133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:25.515444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3c82b-7498-4167-add3-d0093d83f133 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:25.515766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3c82b-7498-4167-add3-d0093d83f133 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:25.516052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3c82b-7498-4167-add3-d0093d83f133 None admin] PATH_INFO: `/v3/services/network` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:25.516709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3c82b-7498-4167-add3-d0093d83f133 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:25.518122 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3c82b-7498-4167-add3-d0093d83f133 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-e0d3c82b-7498-4167-add3-d0093d83f133 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:25.529823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:25.531068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:25.531927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 411/813] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:25 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:25.532838 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3c82b-7498-4167-add3-d0093d83f133 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:25.605258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa74a95-950a-461b-8a69-e82a1a3e6b96 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:25.608606 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:25.638392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3c82b-7498-4167-add3-d0093d83f133 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:25.642457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1ec6d-43ad-4460-ba8e-78e5d5aad3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:25.643120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1ec6d-43ad-4460-ba8e-78e5d5aad3eb None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:25.643395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1ec6d-43ad-4460-ba8e-78e5d5aad3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:25.643676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1ec6d-43ad-4460-ba8e-78e5d5aad3eb None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:25.644407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1ec6d-43ad-4460-ba8e-78e5d5aad3eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:25.644713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1ec6d-43ad-4460-ba8e-78e5d5aad3eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:25.646089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1ec6d-43ad-4460-ba8e-78e5d5aad3eb None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:25.659704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 403/814] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:27:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:25.659704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1ec6d-43ad-4460-ba8e-78e5d5aad3eb None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:25.675759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcf109-fb53-4af8-abfe-34691192b150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:25.676424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcf109-fb53-4af8-abfe-34691192b150 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:25.676707 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcf109-fb53-4af8-abfe-34691192b150 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:25.676991 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcf109-fb53-4af8-abfe-34691192b150 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:25.677793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcf109-fb53-4af8-abfe-34691192b150 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:25.678135 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcf109-fb53-4af8-abfe-34691192b150 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:25.679580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcf109-fb53-4af8-abfe-34691192b150 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:25.692494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 412/815] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:25.693242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcf109-fb53-4af8-abfe-34691192b150 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:25.737471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:25.767408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1ec6d-43ad-4460-ba8e-78e5d5aad3eb None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:25.805440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcf109-fb53-4af8-abfe-34691192b150 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:25.805808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dca85d-0f27-42cd-a4c8-319c74e675ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:25.806522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dca85d-0f27-42cd-a4c8-319c74e675ad None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:25.806810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dca85d-0f27-42cd-a4c8-319c74e675ad None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:25.807090 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dca85d-0f27-42cd-a4c8-319c74e675ad None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:25.807822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dca85d-0f27-42cd-a4c8-319c74e675ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:25.808133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dca85d-0f27-42cd-a4c8-319c74e675ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:25.809509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dca85d-0f27-42cd-a4c8-319c74e675ad None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:25.826137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 404/816] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:25.826137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dca85d-0f27-42cd-a4c8-319c74e675ad None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:25.853448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112ca9e-73f5-4817-97c1-e028f7126b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:25.854514 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:25.854700 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112ca9e-73f5-4817-97c1-e028f7126b19 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:25.854949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112ca9e-73f5-4817-97c1-e028f7126b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:25.855423 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112ca9e-73f5-4817-97c1-e028f7126b19 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:25.856543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112ca9e-73f5-4817-97c1-e028f7126b19 None admin] RBAC: Adding query filter params (name=network) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:25.857040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112ca9e-73f5-4817-97c1-e028f7126b19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:25.858396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112ca9e-73f5-4817-97c1-e028f7126b19 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:25.863173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 413/817] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:25.868319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112ca9e-73f5-4817-97c1-e028f7126b19 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:25.931877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dca85d-0f27-42cd-a4c8-319c74e675ad None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:25.968486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26503bcd-e51a-40c0-99e7-dd1d21227cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:25.969279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26503bcd-e51a-40c0-99e7-dd1d21227cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:25.969564 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26503bcd-e51a-40c0-99e7-dd1d21227cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:25.969844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26503bcd-e51a-40c0-99e7-dd1d21227cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:25.980772 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26503bcd-e51a-40c0-99e7-dd1d21227cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:25.982597 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26503bcd-e51a-40c0-99e7-dd1d21227cf3 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:25.986404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112ca9e-73f5-4817-97c1-e028f7126b19 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:25.989405 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-26503bcd-e51a-40c0-99e7-dd1d21227cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:26.001100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:26.002021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:26.002903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:26.002903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:26.002903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:26.002903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:26.007043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 405/818] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:26.007787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26503bcd-e51a-40c0-99e7-dd1d21227cf3 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:26.024746 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe80bff-ef1e-4cd5-af0a-68d2cd618032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:26.026187 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe80bff-ef1e-4cd5-af0a-68d2cd618032 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:26.026467 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe80bff-ef1e-4cd5-af0a-68d2cd618032 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:26.026742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe80bff-ef1e-4cd5-af0a-68d2cd618032 None admin] PATH_INFO: `/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/e56b0d48e08c4f778a7c509c10edc871/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:26.028727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe80bff-ef1e-4cd5-af0a-68d2cd618032 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff5a41f8ca6438295c512792d2022e4, user_id=e56b0d48e08c4f778a7c509c10edc871, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:26.030193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe80bff-ef1e-4cd5-af0a-68d2cd618032 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:26.045663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 414/819] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:27:25 2022] PUT /identity/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/e56b0d48e08c4f778a7c509c10edc871/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:27:26.046602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe80bff-ef1e-4cd5-af0a-68d2cd618032 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:26.109727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:26.140235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26503bcd-e51a-40c0-99e7-dd1d21227cf3 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:26.185656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f7eeb-70dc-4d24-b7a6-816c38a4cece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:26.186892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f7eeb-70dc-4d24-b7a6-816c38a4cece None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:26.187397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f7eeb-70dc-4d24-b7a6-816c38a4cece None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:26.187977 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f7eeb-70dc-4d24-b7a6-816c38a4cece None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:26.197762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f7eeb-70dc-4d24-b7a6-816c38a4cece None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:26.199382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe80bff-ef1e-4cd5-af0a-68d2cd618032 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:26.199791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f7eeb-70dc-4d24-b7a6-816c38a4cece None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-6d4f7eeb-70dc-4d24-b7a6-816c38a4cece None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:26.218350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:26.219278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:26.220247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:26.220247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:26.220247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:26.220247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:26.220247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:26.220247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:26.220247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:26.220247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:26.222831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 406/820] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:26.223845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f7eeb-70dc-4d24-b7a6-816c38a4cece None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:26.240594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a0ea6-b269-4eed-8cd2-32a360bc13f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:26.241272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a0ea6-b269-4eed-8cd2-32a360bc13f5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:26.241561 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a0ea6-b269-4eed-8cd2-32a360bc13f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:26.242133 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a0ea6-b269-4eed-8cd2-32a360bc13f5 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:26.242868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a0ea6-b269-4eed-8cd2-32a360bc13f5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:26.243179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a0ea6-b269-4eed-8cd2-32a360bc13f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:26.244976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a0ea6-b269-4eed-8cd2-32a360bc13f5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:26.251745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 415/821] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:26.253304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a0ea6-b269-4eed-8cd2-32a360bc13f5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:26.291306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:26.360955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f7eeb-70dc-4d24-b7a6-816c38a4cece None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:26.370326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a0ea6-b269-4eed-8cd2-32a360bc13f5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:26.405474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bf538-df68-41cd-b647-e67d7d0aea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:26.406269 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bf538-df68-41cd-b647-e67d7d0aea77 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:26.406652 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bf538-df68-41cd-b647-e67d7d0aea77 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:26.407025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bf538-df68-41cd-b647-e67d7d0aea77 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:26.407856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bf538-df68-41cd-b647-e67d7d0aea77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:26.408245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bf538-df68-41cd-b647-e67d7d0aea77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:26.409710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bf538-df68-41cd-b647-e67d7d0aea77 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:26.416689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db7880-370c-45de-8d72-08cdd9344a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:26.418074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:26.418349 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db7880-370c-45de-8d72-08cdd9344a44 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:26.418637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db7880-370c-45de-8d72-08cdd9344a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:26.418918 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db7880-370c-45de-8d72-08cdd9344a44 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:26.419728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db7880-370c-45de-8d72-08cdd9344a44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:26.420079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db7880-370c-45de-8d72-08cdd9344a44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:26.421508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db7880-370c-45de-8d72-08cdd9344a44 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:26.433116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 407/822] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:26.433116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bf538-df68-41cd-b647-e67d7d0aea77 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:26.442940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 416/823] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:26.444490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db7880-370c-45de-8d72-08cdd9344a44 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:26.552333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:26.559542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bf538-df68-41cd-b647-e67d7d0aea77 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:26.566642 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db7880-370c-45de-8d72-08cdd9344a44 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:26.602482 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25573319-f4e5-4326-b145-76b8fe9166d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:26.603293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25573319-f4e5-4326-b145-76b8fe9166d7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:26.604138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25573319-f4e5-4326-b145-76b8fe9166d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:26.604417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25573319-f4e5-4326-b145-76b8fe9166d7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:26.606885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae14da7-4a72-4db1-a1ec-b313ae9664bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:26.608369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae14da7-4a72-4db1-a1ec-b313ae9664bf None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:26.608850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae14da7-4a72-4db1-a1ec-b313ae9664bf None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:26.609317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae14da7-4a72-4db1-a1ec-b313ae9664bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:26.610279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae14da7-4a72-4db1-a1ec-b313ae9664bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:26.612132 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae14da7-4a72-4db1-a1ec-b313ae9664bf None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:26.612253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25573319-f4e5-4326-b145-76b8fe9166d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:26.613569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25573319-f4e5-4326-b145-76b8fe9166d7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-25573319-f4e5-4326-b145-76b8fe9166d7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:26.630087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:26.631098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:26.632050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:26.632050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:26.632050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:26.632050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:26.632050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:26.632050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:26.632050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:26.632050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:26.632050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:27:26.632050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:26.636881 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 417/824] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Jul 25 12:27:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:26.637786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25573319-f4e5-4326-b145-76b8fe9166d7 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:26.643497 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 408/825] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:27:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:27:26.644607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae14da7-4a72-4db1-a1ec-b313ae9664bf None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:26.778700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:26.842087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25573319-f4e5-4326-b145-76b8fe9166d7 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:26.849989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae14da7-4a72-4db1-a1ec-b313ae9664bf None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:26.884167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c9cfc-6f22-4213-b7dc-2795590b1c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:26.885593 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c9cfc-6f22-4213-b7dc-2795590b1c71 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:26.885971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c9cfc-6f22-4213-b7dc-2795590b1c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:26.886365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c9cfc-6f22-4213-b7dc-2795590b1c71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:26.887300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c9cfc-6f22-4213-b7dc-2795590b1c71 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=ccf2dedaf7a449d2b51b9e9f3b420091) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:26.887714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c9cfc-6f22-4213-b7dc-2795590b1c71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:26.889261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c9cfc-6f22-4213-b7dc-2795590b1c71 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:26.895999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65284291-8c1b-46a9-98f6-f82342b95762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:26.896877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65284291-8c1b-46a9-98f6-f82342b95762 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:26.896986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 418/826] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:26 2022] GET /identity/v3/role_assignments?user.id=ccf2dedaf7a449d2b51b9e9f3b420091&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:26.897375 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65284291-8c1b-46a9-98f6-f82342b95762 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:26.897818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65284291-8c1b-46a9-98f6-f82342b95762 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:26.898815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65284291-8c1b-46a9-98f6-f82342b95762 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:26.899267 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65284291-8c1b-46a9-98f6-f82342b95762 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:26.900882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65284291-8c1b-46a9-98f6-f82342b95762 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:26.922930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 409/827] 10.222.0.60 () {56 vars in 1149 bytes} [Mon Jul 25 12:27:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:26.928417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c9cfc-6f22-4213-b7dc-2795590b1c71 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:26.948540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:27.042924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c9cfc-6f22-4213-b7dc-2795590b1c71 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:27.080460 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43b2b4-d619-41d6-9760-f7a473b83a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:27.081251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b2b4-d619-41d6-9760-f7a473b83a3a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:27.081530 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b2b4-d619-41d6-9760-f7a473b83a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:27.081799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b2b4-d619-41d6-9760-f7a473b83a3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:27.093023 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43b2b4-d619-41d6-9760-f7a473b83a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:27.094502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43b2b4-d619-41d6-9760-f7a473b83a3a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-1a43b2b4-d619-41d6-9760-f7a473b83a3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:27.112647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:27.113568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:27.114591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:27.114591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:27.114591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:27.114591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:27.114591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 419/828] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:27.124526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65284291-8c1b-46a9-98f6-f82342b95762 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:27.238544 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65284291-8c1b-46a9-98f6-f82342b95762 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:27.250477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:27.269708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d687da-eeca-443d-87f8-792fa81bf89b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:27.270453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d687da-eeca-443d-87f8-792fa81bf89b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:27.270974 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d687da-eeca-443d-87f8-792fa81bf89b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:27.272836 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 420/829] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:27.277951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9abfc-91ac-448d-8363-8f8243594284 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:27.278308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2916c-b391-4c48-b78e-13159723800b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:27.278704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9abfc-91ac-448d-8363-8f8243594284 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:27.279095 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2916c-b391-4c48-b78e-13159723800b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:27.279352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9abfc-91ac-448d-8363-8f8243594284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:27.279502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2916c-b391-4c48-b78e-13159723800b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:27.279820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2916c-b391-4c48-b78e-13159723800b None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:27.280617 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2916c-b391-4c48-b78e-13159723800b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:27.280990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2916c-b391-4c48-b78e-13159723800b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:27.282451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2916c-b391-4c48-b78e-13159723800b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:27.295337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 410/830] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:27.300443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d8f9abfc-91ac-448d-8363-8f8243594284 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:27.300772 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2916c-b391-4c48-b78e-13159723800b None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:27.302080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d8f9abfc-91ac-448d-8363-8f8243594284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:27.378794 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:27.397964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 421/831] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:27.415635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2916c-b391-4c48-b78e-13159723800b None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:27.416826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9abfc-91ac-448d-8363-8f8243594284 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:27.453274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327c92f-204a-4a3a-9cf5-6b7d6bd32762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:27.454264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327c92f-204a-4a3a-9cf5-6b7d6bd32762 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:27.454644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327c92f-204a-4a3a-9cf5-6b7d6bd32762 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:27.455067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327c92f-204a-4a3a-9cf5-6b7d6bd32762 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:27.462034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327c92f-204a-4a3a-9cf5-6b7d6bd32762 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:27.463628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327c92f-204a-4a3a-9cf5-6b7d6bd32762 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-9327c92f-204a-4a3a-9cf5-6b7d6bd32762 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:27.479373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:27.480477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:27.481885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:27.481885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:27.481885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:27.481885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:27.481885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:27.481885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:27.481885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:27.481885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:27.488306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 411/832] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:27.489245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327c92f-204a-4a3a-9cf5-6b7d6bd32762 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:27.520670 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:27.542787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9abfc-91ac-448d-8363-8f8243594284 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:27.589653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6089fb-ab17-472d-9f50-131d956f42df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:27.590879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6089fb-ab17-472d-9f50-131d956f42df None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:27.591439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6089fb-ab17-472d-9f50-131d956f42df None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:27.591729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6089fb-ab17-472d-9f50-131d956f42df None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:27.592400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6089fb-ab17-472d-9f50-131d956f42df None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:27.594808 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6089fb-ab17-472d-9f50-131d956f42df None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:27.596500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327c92f-204a-4a3a-9cf5-6b7d6bd32762 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-ab6089fb-ab17-472d-9f50-131d956f42df None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:27.610822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:27.612027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:27:27.613033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:27.617431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 422/833] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:27:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:27.619655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6089fb-ab17-472d-9f50-131d956f42df None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:27.639910 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b24f7-46a9-40c0-977f-6ffca0afee18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:27.640728 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b24f7-46a9-40c0-977f-6ffca0afee18 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:27.641148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b24f7-46a9-40c0-977f-6ffca0afee18 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:27.641521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b24f7-46a9-40c0-977f-6ffca0afee18 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:27.642485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b24f7-46a9-40c0-977f-6ffca0afee18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:27.642893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b24f7-46a9-40c0-977f-6ffca0afee18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:27.644530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b24f7-46a9-40c0-977f-6ffca0afee18 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:27.662012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 412/834] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:27.663372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b24f7-46a9-40c0-977f-6ffca0afee18 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:27.695602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:27.748188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6089fb-ab17-472d-9f50-131d956f42df None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:27.760799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:27.773944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b24f7-46a9-40c0-977f-6ffca0afee18 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:27.793366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880059cb-6e76-49b5-adc2-2f89b4d374e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:27.794410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880059cb-6e76-49b5-adc2-2f89b4d374e6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:27.794938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880059cb-6e76-49b5-adc2-2f89b4d374e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:27.795417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880059cb-6e76-49b5-adc2-2f89b4d374e6 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:27.796333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880059cb-6e76-49b5-adc2-2f89b4d374e6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:27.796874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880059cb-6e76-49b5-adc2-2f89b4d374e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:27.798597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880059cb-6e76-49b5-adc2-2f89b4d374e6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:27.809351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 423/835] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:27:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:27.811084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880059cb-6e76-49b5-adc2-2f89b4d374e6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:27.816338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d6381-60b8-47e2-b7e2-a1bca1e16578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:27.816822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d6381-60b8-47e2-b7e2-a1bca1e16578 None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:27.816822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d6381-60b8-47e2-b7e2-a1bca1e16578 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:27.817077 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d6381-60b8-47e2-b7e2-a1bca1e16578 None admin] PATH_INFO: `/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/6428e810f10c434ba32d073a542b1de7/roles/d9c5c1adb796486d83edcd6e728a5de9` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:27.832818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d6381-60b8-47e2-b7e2-a1bca1e16578 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff5a41f8ca6438295c512792d2022e4, user_id=6428e810f10c434ba32d073a542b1de7, role_id=d9c5c1adb796486d83edcd6e728a5de9)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:27.834055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d6381-60b8-47e2-b7e2-a1bca1e16578 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:27.850298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 413/836] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:27:27 2022] PUT /identity/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/6428e810f10c434ba32d073a542b1de7/roles/d9c5c1adb796486d83edcd6e728a5de9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:27:27.930541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880059cb-6e76-49b5-adc2-2f89b4d374e6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:27.957874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:27.970263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef28c8-510d-417c-80e6-8d4299db8b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:27.970960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef28c8-510d-417c-80e6-8d4299db8b9e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:27.971244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef28c8-510d-417c-80e6-8d4299db8b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:27.971523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef28c8-510d-417c-80e6-8d4299db8b9e None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:27.972258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef28c8-510d-417c-80e6-8d4299db8b9e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:27.972569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef28c8-510d-417c-80e6-8d4299db8b9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:27.974000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef28c8-510d-417c-80e6-8d4299db8b9e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:27.986076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 424/837] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:27:27 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:28.099892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b723dfd-261b-4cc0-8837-34d8b110024d None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:28.100215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b723dfd-261b-4cc0-8837-34d8b110024d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:28.100505 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b723dfd-261b-4cc0-8837-34d8b110024d None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:28.102103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 414/838] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:28.108686 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e2d9c-5205-403b-b624-6617c2bac136 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:28.108987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e2d9c-5205-403b-b624-6617c2bac136 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:28.109255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e2d9c-5205-403b-b624-6617c2bac136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:28.129646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6c8e2d9c-5205-403b-b624-6617c2bac136 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:28.130916 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6c8e2d9c-5205-403b-b624-6617c2bac136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:28.202970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 425/839] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:28.225880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b723dfd-261b-4cc0-8837-34d8b110024d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:28.334159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b723dfd-261b-4cc0-8837-34d8b110024d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:28.372666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a95fe5-a05c-4667-8eb3-4eab6c38d07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:28.373379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a95fe5-a05c-4667-8eb3-4eab6c38d07d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:28.373660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a95fe5-a05c-4667-8eb3-4eab6c38d07d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:28.373952 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a95fe5-a05c-4667-8eb3-4eab6c38d07d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:28.384540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a95fe5-a05c-4667-8eb3-4eab6c38d07d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:28.391293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-a0a95fe5-a05c-4667-8eb3-4eab6c38d07d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:28.393201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:28.394154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:28.394997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 415/840] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:28.402528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e2d9c-5205-403b-b624-6617c2bac136 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:28.506647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e2d9c-5205-403b-b624-6617c2bac136 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:28.523808 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:28.546628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca731a7-3642-4eca-ab8b-e71a6f6a14bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:28.547305 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca731a7-3642-4eca-ab8b-e71a6f6a14bf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:28.547594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca731a7-3642-4eca-ab8b-e71a6f6a14bf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:28.547862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca731a7-3642-4eca-ab8b-e71a6f6a14bf None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:28.548614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca731a7-3642-4eca-ab8b-e71a6f6a14bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:28.548918 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca731a7-3642-4eca-ab8b-e71a6f6a14bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:28.550301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca731a7-3642-4eca-ab8b-e71a6f6a14bf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:28.559401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 426/841] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:28.567004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a95fe5-a05c-4667-8eb3-4eab6c38d07d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:28.675754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a95fe5-a05c-4667-8eb3-4eab6c38d07d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:28.709717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fff18b-4c00-4668-b36a-7d7551cad706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:28.710540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fff18b-4c00-4668-b36a-7d7551cad706 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:28.710816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fff18b-4c00-4668-b36a-7d7551cad706 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:28.711085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fff18b-4c00-4668-b36a-7d7551cad706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:28.722408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fff18b-4c00-4668-b36a-7d7551cad706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:28.723984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fff18b-4c00-4668-b36a-7d7551cad706 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-c4fff18b-4c00-4668-b36a-7d7551cad706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:28.741800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:28.742800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:28.743724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:28.743724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:28.743724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:28.743724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:28.743724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 416/842] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:28.750680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca731a7-3642-4eca-ab8b-e71a6f6a14bf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:28.855088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca731a7-3642-4eca-ab8b-e71a6f6a14bf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:28.889347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04054452-ab42-4497-9162-72501f881464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:28.890110 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04054452-ab42-4497-9162-72501f881464 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:28.890398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04054452-ab42-4497-9162-72501f881464 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:28.890676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04054452-ab42-4497-9162-72501f881464 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:28.891409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04054452-ab42-4497-9162-72501f881464 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:28.891717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04054452-ab42-4497-9162-72501f881464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:28.893090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04054452-ab42-4497-9162-72501f881464 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:28.906041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 427/843] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:28.911483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fff18b-4c00-4668-b36a-7d7551cad706 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:28.964565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:29.019194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fff18b-4c00-4668-b36a-7d7551cad706 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:29.054143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa0f10-a968-4e49-b15c-2c5af0903713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:29.054946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa0f10-a968-4e49-b15c-2c5af0903713 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:29.055227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa0f10-a968-4e49-b15c-2c5af0903713 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:29.055502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa0f10-a968-4e49-b15c-2c5af0903713 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:29.063845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa0f10-a968-4e49-b15c-2c5af0903713 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:29.065293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa0f10-a968-4e49-b15c-2c5af0903713 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-6caa0f10-a968-4e49-b15c-2c5af0903713 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:29.079783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:29.080760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:29.081716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:29.081716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:29.081716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:29.081716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:29.081716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:29.081716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:29.081716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:29.081716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:29.081716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:27:29.081716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:29.081716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 417/844] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:29.089145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04054452-ab42-4497-9162-72501f881464 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:29.136774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:29.199717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04054452-ab42-4497-9162-72501f881464 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:29.213447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60957ff-f780-460b-bf36-f7f8652c604b None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:29.214101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60957ff-f780-460b-bf36-f7f8652c604b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:29.214641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60957ff-f780-460b-bf36-f7f8652c604b None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:29.216444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 418/845] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:29.221700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f4324-03ef-4623-ad7d-298209dcb9d2 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:29.222243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f4324-03ef-4623-ad7d-298209dcb9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:29.222746 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f4324-03ef-4623-ad7d-298209dcb9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:29.236134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ca4a2-74e9-4672-a545-a1f167e8723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:29.236809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ca4a2-74e9-4672-a545-a1f167e8723f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:29.237094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ca4a2-74e9-4672-a545-a1f167e8723f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:29.237376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ca4a2-74e9-4672-a545-a1f167e8723f None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:29.238252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ca4a2-74e9-4672-a545-a1f167e8723f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:29.238564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ca4a2-74e9-4672-a545-a1f167e8723f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:29.240002 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ca4a2-74e9-4672-a545-a1f167e8723f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:29.243428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b72f4324-03ef-4623-ad7d-298209dcb9d2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:29.245385 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b72f4324-03ef-4623-ad7d-298209dcb9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:29.262949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 428/846] 10.222.0.60 () {56 vars in 1135 bytes} [Mon Jul 25 12:27:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:29.264282 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ca4a2-74e9-4672-a545-a1f167e8723f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:29.333935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 419/847] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:29.347153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f4324-03ef-4623-ad7d-298209dcb9d2 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:29.373048 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:29.382812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ca4a2-74e9-4672-a545-a1f167e8723f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:29.400358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:29.421802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463756cd-255f-41f4-b56e-43ef7aec812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:29.422488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463756cd-255f-41f4-b56e-43ef7aec812b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:29.422775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463756cd-255f-41f4-b56e-43ef7aec812b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:29.423057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463756cd-255f-41f4-b56e-43ef7aec812b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:29.473367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f4324-03ef-4623-ad7d-298209dcb9d2 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:29.510781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44393d1a-6c65-4ab1-8368-3aa1f69aa325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:29.511964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44393d1a-6c65-4ab1-8368-3aa1f69aa325 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:29.512441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44393d1a-6c65-4ab1-8368-3aa1f69aa325 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:29.512922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44393d1a-6c65-4ab1-8368-3aa1f69aa325 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:29.513767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44393d1a-6c65-4ab1-8368-3aa1f69aa325 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:29.515412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44393d1a-6c65-4ab1-8368-3aa1f69aa325 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:29.516890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463756cd-255f-41f4-b56e-43ef7aec812b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:29.518576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463756cd-255f-41f4-b56e-43ef7aec812b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-44393d1a-6c65-4ab1-8368-3aa1f69aa325 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:29.528397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:29.529312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:27:29.530166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:29.537650 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 420/848] 10.222.0.60 () {56 vars in 1132 bytes} [Mon Jul 25 12:27:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:29.538744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44393d1a-6c65-4ab1-8368-3aa1f69aa325 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:29.620353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:29.632497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 429/849] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:27:29.633527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463756cd-255f-41f4-b56e-43ef7aec812b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:29.652335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44393d1a-6c65-4ab1-8368-3aa1f69aa325 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:29.655176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:29.688849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aafac5-f517-41e0-ace5-1c7467e7ba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:29.689541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aafac5-f517-41e0-ace5-1c7467e7ba6d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:29.689820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aafac5-f517-41e0-ace5-1c7467e7ba6d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:29.690136 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aafac5-f517-41e0-ace5-1c7467e7ba6d None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:29.690849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aafac5-f517-41e0-ace5-1c7467e7ba6d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:29.691154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aafac5-f517-41e0-ace5-1c7467e7ba6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:29.692512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aafac5-f517-41e0-ace5-1c7467e7ba6d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:29.702688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 421/850] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Jul 25 12:27:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:29.703589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aafac5-f517-41e0-ace5-1c7467e7ba6d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:29.748714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463756cd-255f-41f4-b56e-43ef7aec812b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:29.786418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d44a7-ebb5-4dba-b454-cc76f3a5ab2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:29.787707 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d44a7-ebb5-4dba-b454-cc76f3a5ab2c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:29.788197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d44a7-ebb5-4dba-b454-cc76f3a5ab2c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:29.788745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d44a7-ebb5-4dba-b454-cc76f3a5ab2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:29.800233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d44a7-ebb5-4dba-b454-cc76f3a5ab2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:29.802149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d44a7-ebb5-4dba-b454-cc76f3a5ab2c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:29.807265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aafac5-f517-41e0-ace5-1c7467e7ba6d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-057d44a7-ebb5-4dba-b454-cc76f3a5ab2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:29.822922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:29.823860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:29.824752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:29.824752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:29.824752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:29.824752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:29.827153 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 430/851] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:29.833385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d44a7-ebb5-4dba-b454-cc76f3a5ab2c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:29.851171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4680d5-2a5b-4531-bc6f-d004dd0a8cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:29.851843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4680d5-2a5b-4531-bc6f-d004dd0a8cec None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:29.852130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4680d5-2a5b-4531-bc6f-d004dd0a8cec None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:29.852425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4680d5-2a5b-4531-bc6f-d004dd0a8cec None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:29.853126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4680d5-2a5b-4531-bc6f-d004dd0a8cec None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:29.853463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4680d5-2a5b-4531-bc6f-d004dd0a8cec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:29.855174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4680d5-2a5b-4531-bc6f-d004dd0a8cec None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:29.870476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 422/852] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Jul 25 12:27:29 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:29.899142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:29.962074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d44a7-ebb5-4dba-b454-cc76f3a5ab2c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:30.002790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cf7e1-e8f2-4081-be4d-21b785b87a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:30.003858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cf7e1-e8f2-4081-be4d-21b785b87a9c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:30.004351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cf7e1-e8f2-4081-be4d-21b785b87a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:30.004862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cf7e1-e8f2-4081-be4d-21b785b87a9c None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:30.005833 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cf7e1-e8f2-4081-be4d-21b785b87a9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:30.006429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cf7e1-e8f2-4081-be4d-21b785b87a9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:30.008044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cf7e1-e8f2-4081-be4d-21b785b87a9c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:30.021021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 431/853] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:30.026875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4680d5-2a5b-4531-bc6f-d004dd0a8cec None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:30.048234 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52a401-5092-4eb5-ac84-546ebb52d12b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:30.048528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52a401-5092-4eb5-ac84-546ebb52d12b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:30.048794 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52a401-5092-4eb5-ac84-546ebb52d12b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:30.050448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 432/854] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:30.055556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4738d7-f8a7-48a2-83f5-aceb90d61980 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:30.055850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4738d7-f8a7-48a2-83f5-aceb90d61980 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:30.056129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4738d7-f8a7-48a2-83f5-aceb90d61980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:30.063098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:30.076433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6c4738d7-f8a7-48a2-83f5-aceb90d61980 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:30.077644 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6c4738d7-f8a7-48a2-83f5-aceb90d61980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:30.140009 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4680d5-2a5b-4531-bc6f-d004dd0a8cec None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:30.153972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 433/855] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:30.153972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b9c8-e156-4e9a-89af-5527cb152d11 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:30.153972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b9c8-e156-4e9a-89af-5527cb152d11 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:30.153972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b9c8-e156-4e9a-89af-5527cb152d11 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:30.153972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 434/856] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:30.156463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9640faf-3072-4dc5-92c3-7e81ad9ab972 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:30.156895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9640faf-3072-4dc5-92c3-7e81ad9ab972 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:30.157314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9640faf-3072-4dc5-92c3-7e81ad9ab972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:30.179516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c9640faf-3072-4dc5-92c3-7e81ad9ab972 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:30.180714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c9640faf-3072-4dc5-92c3-7e81ad9ab972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:30.195233 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b307302-5f17-48c8-b913-0f6b9cb65ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:30.196310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b307302-5f17-48c8-b913-0f6b9cb65ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:30.196590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b307302-5f17-48c8-b913-0f6b9cb65ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:30.196863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b307302-5f17-48c8-b913-0f6b9cb65ac3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:30.206014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b307302-5f17-48c8-b913-0f6b9cb65ac3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:30.207956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b307302-5f17-48c8-b913-0f6b9cb65ac3 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:30.213610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-2b307302-5f17-48c8-b913-0f6b9cb65ac3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:30.226719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:30.227641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:30.229094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:30.229094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:30.229094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:30.229094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:30.229094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:30.229094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:30.229094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:30.229094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:30.230248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 423/857] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:30.231086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b307302-5f17-48c8-b913-0f6b9cb65ac3 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:30.264458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 435/858] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:30.265765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9640faf-3072-4dc5-92c3-7e81ad9ab972 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:30.326935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:30.336161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b307302-5f17-48c8-b913-0f6b9cb65ac3 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:30.374474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab3e1b-6b24-44c5-ba80-ec6c2a5b1306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:30.375265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab3e1b-6b24-44c5-ba80-ec6c2a5b1306 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:30.375551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab3e1b-6b24-44c5-ba80-ec6c2a5b1306 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:30.375818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab3e1b-6b24-44c5-ba80-ec6c2a5b1306 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:30.385578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9640faf-3072-4dc5-92c3-7e81ad9ab972 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:30.385776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab3e1b-6b24-44c5-ba80-ec6c2a5b1306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-efab3e1b-6b24-44c5-ba80-ec6c2a5b1306 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:30.393492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:30.394510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:30.395339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 424/859] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:27:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:30.396027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab3e1b-6b24-44c5-ba80-ec6c2a5b1306 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:30.424907 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce95a1-4205-41e1-a09a-b55b33712794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:30.425586 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce95a1-4205-41e1-a09a-b55b33712794 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:30.425890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce95a1-4205-41e1-a09a-b55b33712794 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:30.426172 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce95a1-4205-41e1-a09a-b55b33712794 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:30.426966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce95a1-4205-41e1-a09a-b55b33712794 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:30.427276 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce95a1-4205-41e1-a09a-b55b33712794 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:30.428718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce95a1-4205-41e1-a09a-b55b33712794 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:30.441240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 436/860] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:30.442079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce95a1-4205-41e1-a09a-b55b33712794 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:30.476429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:30.506376 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab3e1b-6b24-44c5-ba80-ec6c2a5b1306 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:30.541761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09446038-d9d9-4c62-a53d-8d2388d655b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:30.542452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09446038-d9d9-4c62-a53d-8d2388d655b2 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:30.542731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09446038-d9d9-4c62-a53d-8d2388d655b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:30.543006 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09446038-d9d9-4c62-a53d-8d2388d655b2 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:30.544000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09446038-d9d9-4c62-a53d-8d2388d655b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:30.545068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09446038-d9d9-4c62-a53d-8d2388d655b2 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:30.557493 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce95a1-4205-41e1-a09a-b55b33712794 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:30.572802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 425/861] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:27:30 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:27:30.572802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09446038-d9d9-4c62-a53d-8d2388d655b2 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:30.580030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:30.621626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59401633-1040-42cb-b117-decbe6b4acc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:30.622458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59401633-1040-42cb-b117-decbe6b4acc7 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:30.622816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59401633-1040-42cb-b117-decbe6b4acc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:30.623163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59401633-1040-42cb-b117-decbe6b4acc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:30.624017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59401633-1040-42cb-b117-decbe6b4acc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:30.624393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59401633-1040-42cb-b117-decbe6b4acc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:30.625900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59401633-1040-42cb-b117-decbe6b4acc7 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:30.637223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 437/862] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:27:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:30.643226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59401633-1040-42cb-b117-decbe6b4acc7 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:30.702748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09446038-d9d9-4c62-a53d-8d2388d655b2 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:30.743509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56aa1d7-4379-4f9d-8321-338763b7b8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:30.744210 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56aa1d7-4379-4f9d-8321-338763b7b8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:30.744493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56aa1d7-4379-4f9d-8321-338763b7b8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:30.744772 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56aa1d7-4379-4f9d-8321-338763b7b8f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:30.745606 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56aa1d7-4379-4f9d-8321-338763b7b8f8 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=e56b0d48e08c4f778a7c509c10edc871) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:30.745951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56aa1d7-4379-4f9d-8321-338763b7b8f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:30.747387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56aa1d7-4379-4f9d-8321-338763b7b8f8 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:30.753455 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 426/863] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:30 2022] GET /identity/v3/role_assignments?user.id=e56b0d48e08c4f778a7c509c10edc871&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:30.768456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:30.790082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59401633-1040-42cb-b117-decbe6b4acc7 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:30.832625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4533f8-efe2-44ea-82e6-d326226a44b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:30.833355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4533f8-efe2-44ea-82e6-d326226a44b4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:30.833636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4533f8-efe2-44ea-82e6-d326226a44b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:30.834013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4533f8-efe2-44ea-82e6-d326226a44b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:30.847342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4533f8-efe2-44ea-82e6-d326226a44b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:30.848883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4533f8-efe2-44ea-82e6-d326226a44b4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-ce4533f8-efe2-44ea-82e6-d326226a44b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:30.868272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:30.869243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:30.870689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:30.870689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:30.870689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:30.870689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:30.873553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 438/864] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:30.875748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56aa1d7-4379-4f9d-8321-338763b7b8f8 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:30.958997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:30.986706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56aa1d7-4379-4f9d-8321-338763b7b8f8 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:31.024107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358534e-dc46-4e7d-bd2e-71d295155bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:31.024828 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358534e-dc46-4e7d-bd2e-71d295155bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:31.025142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358534e-dc46-4e7d-bd2e-71d295155bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:31.025423 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358534e-dc46-4e7d-bd2e-71d295155bd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:31.026178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358534e-dc46-4e7d-bd2e-71d295155bd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:31.026484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358534e-dc46-4e7d-bd2e-71d295155bd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:31.027849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358534e-dc46-4e7d-bd2e-71d295155bd0 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:31.041123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 427/865] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:31.046270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4533f8-efe2-44ea-82e6-d326226a44b4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:31.152052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4533f8-efe2-44ea-82e6-d326226a44b4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:31.186928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab81a1-1ca0-48f8-bfc9-6f4204d7a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:31.188175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab81a1-1ca0-48f8-bfc9-6f4204d7a77b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:31.188175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab81a1-1ca0-48f8-bfc9-6f4204d7a77b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:31.188428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab81a1-1ca0-48f8-bfc9-6f4204d7a77b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:31.196224 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab81a1-1ca0-48f8-bfc9-6f4204d7a77b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:31.198515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab81a1-1ca0-48f8-bfc9-6f4204d7a77b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:31.200163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-baab81a1-1ca0-48f8-bfc9-6f4204d7a77b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:31.213662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:31.215671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:31.217924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:31.217924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:31.217924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:31.217924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:31.217924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:31.217924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:31.217924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:31.217924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:31.217924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:27:31.217924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:31.217924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 439/866] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Jul 25 12:27:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:31.221877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358534e-dc46-4e7d-bd2e-71d295155bd0 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:31.323475 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358534e-dc46-4e7d-bd2e-71d295155bd0 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:31.359122 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310a621-e34d-4d1a-8d4f-aaf2b39f0e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:31.359717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310a621-e34d-4d1a-8d4f-aaf2b39f0e15 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:31.360001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310a621-e34d-4d1a-8d4f-aaf2b39f0e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:31.360279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310a621-e34d-4d1a-8d4f-aaf2b39f0e15 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:31.361120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310a621-e34d-4d1a-8d4f-aaf2b39f0e15 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:31.361427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310a621-e34d-4d1a-8d4f-aaf2b39f0e15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:31.362882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310a621-e34d-4d1a-8d4f-aaf2b39f0e15 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:31.365584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:31.382801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 428/867] 10.222.0.60 () {56 vars in 1149 bytes} [Mon Jul 25 12:27:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:31.388533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab81a1-1ca0-48f8-bfc9-6f4204d7a77b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:31.491019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab81a1-1ca0-48f8-bfc9-6f4204d7a77b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:31.525072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545e86c-e090-42e1-a33b-60d711e392b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:31.525750 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545e86c-e090-42e1-a33b-60d711e392b9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:31.526120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545e86c-e090-42e1-a33b-60d711e392b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:31.526397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545e86c-e090-42e1-a33b-60d711e392b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:31.595344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545e86c-e090-42e1-a33b-60d711e392b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:31.596790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545e86c-e090-42e1-a33b-60d711e392b9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:31.598634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:31.699887 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 440/868] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:31 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:27:31.705349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310a621-e34d-4d1a-8d4f-aaf2b39f0e15 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:31.815001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310a621-e34d-4d1a-8d4f-aaf2b39f0e15 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:31.850988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebcaca-092e-4e1d-838a-d1d5cad9dab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:31.851831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebcaca-092e-4e1d-838a-d1d5cad9dab4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:31.852118 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebcaca-092e-4e1d-838a-d1d5cad9dab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:31.852389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebcaca-092e-4e1d-838a-d1d5cad9dab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:31.863874 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ebcaca-092e-4e1d-838a-d1d5cad9dab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:31.865313 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ebcaca-092e-4e1d-838a-d1d5cad9dab4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-30ebcaca-092e-4e1d-838a-d1d5cad9dab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:31.882858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:31.883751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:31.885303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:31.885303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:31.885303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:31.885303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:31.885303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 429/869] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:31.889606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545e86c-e090-42e1-a33b-60d711e392b9 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:31.942722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f5f9a-368e-44c8-a13f-f0460c133e1b None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:31.943075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f5f9a-368e-44c8-a13f-f0460c133e1b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:31.943287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f5f9a-368e-44c8-a13f-f0460c133e1b None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:31.944811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 430/870] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:31.950151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923f7bd-8a07-44b7-ba9e-b0d4f7b6e90c None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:31.950673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923f7bd-8a07-44b7-ba9e-b0d4f7b6e90c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:31.951194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923f7bd-8a07-44b7-ba9e-b0d4f7b6e90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:31.971537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-6923f7bd-8a07-44b7-ba9e-b0d4f7b6e90c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:31.973148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6923f7bd-8a07-44b7-ba9e-b0d4f7b6e90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:31.989166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:31.993214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545e86c-e090-42e1-a33b-60d711e392b9 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:32.023749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:32.030223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a98f8-9b2b-4c21-a968-2315b308c83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:32.030890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a98f8-9b2b-4c21-a968-2315b308c83a None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:32.031176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a98f8-9b2b-4c21-a968-2315b308c83a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:32.031454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a98f8-9b2b-4c21-a968-2315b308c83a None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:32.032174 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a98f8-9b2b-4c21-a968-2315b308c83a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:32.032477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a98f8-9b2b-4c21-a968-2315b308c83a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:32.033847 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a98f8-9b2b-4c21-a968-2315b308c83a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:32.050713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 441/871] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:32.051574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a98f8-9b2b-4c21-a968-2315b308c83a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:32.062022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 431/872] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:31 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:32.080714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923f7bd-8a07-44b7-ba9e-b0d4f7b6e90c None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:32.168927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a98f8-9b2b-4c21-a968-2315b308c83a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:32.201745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923f7bd-8a07-44b7-ba9e-b0d4f7b6e90c None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:32.216894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa555ae-0fd0-4127-b180-bd422dc9e02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:32.218246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa555ae-0fd0-4127-b180-bd422dc9e02f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:32.218796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa555ae-0fd0-4127-b180-bd422dc9e02f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:32.219075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa555ae-0fd0-4127-b180-bd422dc9e02f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:32.228403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa555ae-0fd0-4127-b180-bd422dc9e02f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:32.230290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa555ae-0fd0-4127-b180-bd422dc9e02f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:32.238624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035e0c0-e89b-4e1a-be7e-42a6e0fc04e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:32.239409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035e0c0-e89b-4e1a-be7e-42a6e0fc04e2 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:32.239786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035e0c0-e89b-4e1a-be7e-42a6e0fc04e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:32.240157 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035e0c0-e89b-4e1a-be7e-42a6e0fc04e2 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:32.240974 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035e0c0-e89b-4e1a-be7e-42a6e0fc04e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:32.242450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035e0c0-e89b-4e1a-be7e-42a6e0fc04e2 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-4fa555ae-0fd0-4127-b180-bd422dc9e02f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:32.247701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:32.249151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:32.250714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:32.250714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:32.250714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:32.250714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:32.250714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:32.250714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:32.250714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:32.250714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:32.250714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 442/873] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:32.255913 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa555ae-0fd0-4127-b180-bd422dc9e02f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:32.263277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 432/874] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:27:32 2022] POST /identity/v3/services => generated 269 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:27:32.279699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:32.375198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa555ae-0fd0-4127-b180-bd422dc9e02f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:32.418981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf70937-6d9d-4072-b628-144e937e6962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:32.419759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf70937-6d9d-4072-b628-144e937e6962 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:32.420113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf70937-6d9d-4072-b628-144e937e6962 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:32.420452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf70937-6d9d-4072-b628-144e937e6962 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:32.421313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf70937-6d9d-4072-b628-144e937e6962 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:32.421692 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf70937-6d9d-4072-b628-144e937e6962 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:32.423243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf70937-6d9d-4072-b628-144e937e6962 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:32.436867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 443/875] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:32.442473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035e0c0-e89b-4e1a-be7e-42a6e0fc04e2 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:32.480380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:32.551466 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035e0c0-e89b-4e1a-be7e-42a6e0fc04e2 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:32.587244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a603ec-24fa-43b6-8a28-cc9ef9d4beae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:32.588771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a603ec-24fa-43b6-8a28-cc9ef9d4beae None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:32.589193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a603ec-24fa-43b6-8a28-cc9ef9d4beae None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:32.589608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a603ec-24fa-43b6-8a28-cc9ef9d4beae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:32.590640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a603ec-24fa-43b6-8a28-cc9ef9d4beae None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=6428e810f10c434ba32d073a542b1de7) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:32.591091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a603ec-24fa-43b6-8a28-cc9ef9d4beae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:32.592666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a603ec-24fa-43b6-8a28-cc9ef9d4beae None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:32.606523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 433/876] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:32 2022] GET /identity/v3/role_assignments?user.id=6428e810f10c434ba32d073a542b1de7&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:32.758291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051867e-796b-483d-9b2e-e8815179a3f7 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:32.758802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051867e-796b-483d-9b2e-e8815179a3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:32.759222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051867e-796b-483d-9b2e-e8815179a3f7 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:32.761008 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 444/877] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:32.766288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd6c6b-2add-45c1-a9f2-e74932e03922 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:32.766761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd6c6b-2add-45c1-a9f2-e74932e03922 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:32.767200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd6c6b-2add-45c1-a9f2-e74932e03922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:32.789062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-73bd6c6b-2add-45c1-a9f2-e74932e03922 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:32.790474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-73bd6c6b-2add-45c1-a9f2-e74932e03922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:32.861644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 434/878] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:32.883660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051867e-796b-483d-9b2e-e8815179a3f7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:32.917105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6c027-f675-4660-b4f0-3abb59819d7b None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:32.917738 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6c027-f675-4660-b4f0-3abb59819d7b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:32.918312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6c027-f675-4660-b4f0-3abb59819d7b None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:32.920249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 435/879] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:32.925844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff7b5a-d3d4-4af7-b3c9-7adedaa32da1 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:32.926173 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff7b5a-d3d4-4af7-b3c9-7adedaa32da1 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:32.926588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff7b5a-d3d4-4af7-b3c9-7adedaa32da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:32.945464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-6dff7b5a-d3d4-4af7-b3c9-7adedaa32da1 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:32.946696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6dff7b5a-d3d4-4af7-b3c9-7adedaa32da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:32.970513 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:32.976062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:33.002391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051867e-796b-483d-9b2e-e8815179a3f7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:33.034075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 436/880] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jul 25 12:27:33.050598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186d6c4-db88-40b9-adf2-a46229c1578c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:33.052626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff7b5a-d3d4-4af7-b3c9-7adedaa32da1 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:33.054216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186d6c4-db88-40b9-adf2-a46229c1578c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:33.054941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186d6c4-db88-40b9-adf2-a46229c1578c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:33.055433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186d6c4-db88-40b9-adf2-a46229c1578c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:33.057238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186d6c4-db88-40b9-adf2-a46229c1578c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:33.058925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186d6c4-db88-40b9-adf2-a46229c1578c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-3186d6c4-db88-40b9-adf2-a46229c1578c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:33.074712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:33.075718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:27:33.076596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:33.079824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 445/881] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:27:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:33.086460 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186d6c4-db88-40b9-adf2-a46229c1578c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:33.159997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff7b5a-d3d4-4af7-b3c9-7adedaa32da1 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:33.195323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f81a48-5b01-4f31-92dd-cf39156d50ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:33.196328 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f81a48-5b01-4f31-92dd-cf39156d50ee None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:33.196761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f81a48-5b01-4f31-92dd-cf39156d50ee None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:33.197725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f81a48-5b01-4f31-92dd-cf39156d50ee None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:33.198422 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186d6c4-db88-40b9-adf2-a46229c1578c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:33.198762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f81a48-5b01-4f31-92dd-cf39156d50ee None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:33.200274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f81a48-5b01-4f31-92dd-cf39156d50ee None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-a4f81a48-5b01-4f31-92dd-cf39156d50ee None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:33.213172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:33.214234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:33.215061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:33.222687 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 437/882] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Jul 25 12:27:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:33.224153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f81a48-5b01-4f31-92dd-cf39156d50ee None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:33.236013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132bceb-6e1c-43d1-928c-37d76672c34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:33.236525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132bceb-6e1c-43d1-928c-37d76672c34c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:33.236905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132bceb-6e1c-43d1-928c-37d76672c34c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:33.237252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132bceb-6e1c-43d1-928c-37d76672c34c None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:33.238109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132bceb-6e1c-43d1-928c-37d76672c34c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:33.238727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132bceb-6e1c-43d1-928c-37d76672c34c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:33.240195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132bceb-6e1c-43d1-928c-37d76672c34c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:33.250796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 446/883] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:27:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:33.251956 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132bceb-6e1c-43d1-928c-37d76672c34c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:33.255475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:33.329033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f81a48-5b01-4f31-92dd-cf39156d50ee None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:33.342254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:33.362341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132bceb-6e1c-43d1-928c-37d76672c34c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:33.366618 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16696a7e-bdd3-410e-a455-0f94d3454a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:33.367315 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16696a7e-bdd3-410e-a455-0f94d3454a28 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:33.367597 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16696a7e-bdd3-410e-a455-0f94d3454a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:33.367882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16696a7e-bdd3-410e-a455-0f94d3454a28 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:33.368766 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16696a7e-bdd3-410e-a455-0f94d3454a28 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:33.369425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16696a7e-bdd3-410e-a455-0f94d3454a28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:33.371144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16696a7e-bdd3-410e-a455-0f94d3454a28 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:33.377178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 438/884] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Jul 25 12:27:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:33.382838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16696a7e-bdd3-410e-a455-0f94d3454a28 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:33.402139 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5facf-3483-4112-9fee-8db5c2e886c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:33.403094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5facf-3483-4112-9fee-8db5c2e886c0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:33.403525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5facf-3483-4112-9fee-8db5c2e886c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:33.403944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5facf-3483-4112-9fee-8db5c2e886c0 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:33.404912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5facf-3483-4112-9fee-8db5c2e886c0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:33.405442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5facf-3483-4112-9fee-8db5c2e886c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:33.407125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5facf-3483-4112-9fee-8db5c2e886c0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:33.417442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 447/885] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:27:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:33.418842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5facf-3483-4112-9fee-8db5c2e886c0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:33.492224 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:33.502981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:33.519156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16696a7e-bdd3-410e-a455-0f94d3454a28 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:33.534318 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5facf-3483-4112-9fee-8db5c2e886c0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:33.560454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6799e40-a52d-4b8d-9620-4e2de53f681b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:33.561131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6799e40-a52d-4b8d-9620-4e2de53f681b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:33.561415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6799e40-a52d-4b8d-9620-4e2de53f681b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:33.561697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6799e40-a52d-4b8d-9620-4e2de53f681b None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:33.562761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6799e40-a52d-4b8d-9620-4e2de53f681b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:33.563074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6799e40-a52d-4b8d-9620-4e2de53f681b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:33.564723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6799e40-a52d-4b8d-9620-4e2de53f681b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:33.570983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 439/886] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Jul 25 12:27:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:33.578839 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947fd61-f79c-40b8-8e10-6c7e2762f33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:33.579857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947fd61-f79c-40b8-8e10-6c7e2762f33c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:33.580317 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947fd61-f79c-40b8-8e10-6c7e2762f33c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:33.580724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947fd61-f79c-40b8-8e10-6c7e2762f33c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:33.581787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947fd61-f79c-40b8-8e10-6c7e2762f33c None admin] RBAC: Adding query filter params (interface=public, service_id=0a34e5a3cf554525aea6910a6bdec6de, region_id=RegionOne) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:33.582334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947fd61-f79c-40b8-8e10-6c7e2762f33c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:33.583922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947fd61-f79c-40b8-8e10-6c7e2762f33c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:33.593592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 448/887] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Jul 25 12:27:33 2022] GET /identity/v3/endpoints?service_id=0a34e5a3cf554525aea6910a6bdec6de&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:33.599680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6799e40-a52d-4b8d-9620-4e2de53f681b None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:33.716402 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6799e40-a52d-4b8d-9620-4e2de53f681b None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:33.754210 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdf1ae-82d8-4359-b0a3-01ef8e0a36c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:33.754921 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf1ae-82d8-4359-b0a3-01ef8e0a36c5 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:33.755208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf1ae-82d8-4359-b0a3-01ef8e0a36c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:33.755486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf1ae-82d8-4359-b0a3-01ef8e0a36c5 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:33.756180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdf1ae-82d8-4359-b0a3-01ef8e0a36c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:33.757551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdf1ae-82d8-4359-b0a3-01ef8e0a36c5 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:33.763951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 440/888] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Jul 25 12:27:33 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:27:34.400819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7af280-9635-4691-ab4e-284c874fa5ef None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:34.401328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7af280-9635-4691-ab4e-284c874fa5ef None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:34.401435 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7af280-9635-4691-ab4e-284c874fa5ef None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:34.402962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 449/889] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:34.408403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55a25d-bd99-4af4-9419-1ad99db33a16 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:34.408837 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55a25d-bd99-4af4-9419-1ad99db33a16 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:34.409018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55a25d-bd99-4af4-9419-1ad99db33a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:34.427238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-7e55a25d-bd99-4af4-9419-1ad99db33a16 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:34.428406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7e55a25d-bd99-4af4-9419-1ad99db33a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:34.439370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:34.500262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 441/890] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:34.522141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7af280-9635-4691-ab4e-284c874fa5ef None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:34.630449 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7af280-9635-4691-ab4e-284c874fa5ef None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:34.660454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:34.667453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9d75f-f6a9-4af8-9d84-8c8c44b4192b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:34.668463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9d75f-f6a9-4af8-9d84-8c8c44b4192b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:34.668753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9d75f-f6a9-4af8-9d84-8c8c44b4192b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:34.669028 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9d75f-f6a9-4af8-9d84-8c8c44b4192b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:34.669685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9d75f-f6a9-4af8-9d84-8c8c44b4192b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:34.671109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9d75f-f6a9-4af8-9d84-8c8c44b4192b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-56f9d75f-f6a9-4af8-9d84-8c8c44b4192b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:34.683747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:34.684675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:34.685535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 450/891] 10.222.0.60 () {56 vars in 1132 bytes} [Mon Jul 25 12:27:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:34.693105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55a25d-bd99-4af4-9419-1ad99db33a16 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:34.727816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c895c2-6882-4912-8580-48482a1ae592 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:34.728147 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c895c2-6882-4912-8580-48482a1ae592 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:34.728908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c895c2-6882-4912-8580-48482a1ae592 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:34.731063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 451/892] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:34.736632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13906b9-ce15-4cd5-a62e-3575b47762e2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:34.736632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13906b9-ce15-4cd5-a62e-3575b47762e2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:34.736866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13906b9-ce15-4cd5-a62e-3575b47762e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:34.755514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c13906b9-ce15-4cd5-a62e-3575b47762e2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:34.756682 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c13906b9-ce15-4cd5-a62e-3575b47762e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:34.811901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55a25d-bd99-4af4-9419-1ad99db33a16 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:34.830524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 452/893] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:34.855067 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13906b9-ce15-4cd5-a62e-3575b47762e2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:34.856580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da05ae-203c-4c26-8f10-6b920a486fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:34.857519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da05ae-203c-4c26-8f10-6b920a486fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:34.858068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da05ae-203c-4c26-8f10-6b920a486fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:34.858574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da05ae-203c-4c26-8f10-6b920a486fc4 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:34.859303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da05ae-203c-4c26-8f10-6b920a486fc4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:34.859622 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da05ae-203c-4c26-8f10-6b920a486fc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:34.861004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da05ae-203c-4c26-8f10-6b920a486fc4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:34.869596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 442/894] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Jul 25 12:27:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:34.871582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da05ae-203c-4c26-8f10-6b920a486fc4 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:34.883476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:34.989934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13906b9-ce15-4cd5-a62e-3575b47762e2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:34.995422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da05ae-203c-4c26-8f10-6b920a486fc4 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:35.021257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:35.034246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb24096-deea-4edf-a351-1971d58cd6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:35.035324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb24096-deea-4edf-a351-1971d58cd6cb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.035604 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb24096-deea-4edf-a351-1971d58cd6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.035878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb24096-deea-4edf-a351-1971d58cd6cb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.039595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e7672-a3c8-4e83-8cf8-96e6b5490b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:35.040493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e7672-a3c8-4e83-8cf8-96e6b5490b86 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.041551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e7672-a3c8-4e83-8cf8-96e6b5490b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.042078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e7672-a3c8-4e83-8cf8-96e6b5490b86 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.042989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e7672-a3c8-4e83-8cf8-96e6b5490b86 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:35.043619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e7672-a3c8-4e83-8cf8-96e6b5490b86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:35.045317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e7672-a3c8-4e83-8cf8-96e6b5490b86 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:35.047230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb24096-deea-4edf-a351-1971d58cd6cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:35.051107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 443/895] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Jul 25 12:27:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:35.057202 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e7672-a3c8-4e83-8cf8-96e6b5490b86 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-9eb24096-deea-4edf-a351-1971d58cd6cb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:35.061822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:35.062766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:35.063616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:35.063616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:35.063616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:35.063616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:35.063616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:35.063616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:35.063616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:35.063616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:35.063616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:35.063616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:35.063616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:27:35.063616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:35.067951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 453/896] 10.222.0.60 () {56 vars in 1120 bytes} [Mon Jul 25 12:27:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:35.068799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb24096-deea-4edf-a351-1971d58cd6cb None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:35.142600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:35.178741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e7672-a3c8-4e83-8cf8-96e6b5490b86 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:35.209331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb24096-deea-4edf-a351-1971d58cd6cb None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:35.222007 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc42209-0e9a-4ad7-9222-683f8d4502bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:35.223037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc42209-0e9a-4ad7-9222-683f8d4502bd None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.223548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc42209-0e9a-4ad7-9222-683f8d4502bd None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.224039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc42209-0e9a-4ad7-9222-683f8d4502bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.225132 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc42209-0e9a-4ad7-9222-683f8d4502bd None admin] RBAC: Adding query filter params (interface=public, service_id=7914e4c2e84b4f2c8057efb481dca4a1, region_id=RegionOne) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:35.225744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc42209-0e9a-4ad7-9222-683f8d4502bd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:35.227400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc42209-0e9a-4ad7-9222-683f8d4502bd None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:35.234387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 444/897] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Jul 25 12:27:35 2022] GET /identity/v3/endpoints?service_id=7914e4c2e84b4f2c8057efb481dca4a1&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:35.239738 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc42209-0e9a-4ad7-9222-683f8d4502bd None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:35.254931 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ba8a1-67c6-4007-a070-65480fad8f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:35.255585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ba8a1-67c6-4007-a070-65480fad8f50 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.256215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ba8a1-67c6-4007-a070-65480fad8f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.256342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ba8a1-67c6-4007-a070-65480fad8f50 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.257038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ba8a1-67c6-4007-a070-65480fad8f50 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:35.257346 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ba8a1-67c6-4007-a070-65480fad8f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:35.258900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ba8a1-67c6-4007-a070-65480fad8f50 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:35.272657 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 454/898] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Jul 25 12:27:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:35.274140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ba8a1-67c6-4007-a070-65480fad8f50 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:35.326372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:35.376524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc42209-0e9a-4ad7-9222-683f8d4502bd None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:35.385680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ba8a1-67c6-4007-a070-65480fad8f50 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:35.422583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873aefff-af53-44cf-8499-faf51048b359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:35.423616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873aefff-af53-44cf-8499-faf51048b359 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.424093 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873aefff-af53-44cf-8499-faf51048b359 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.424470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873aefff-af53-44cf-8499-faf51048b359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.426618 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3d654-3bd9-4b7e-a8d0-ec17988bf89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:35.428144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3d654-3bd9-4b7e-a8d0-ec17988bf89c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.428637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3d654-3bd9-4b7e-a8d0-ec17988bf89c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.429051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3d654-3bd9-4b7e-a8d0-ec17988bf89c None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.429883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3d654-3bd9-4b7e-a8d0-ec17988bf89c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:35.432032 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3d654-3bd9-4b7e-a8d0-ec17988bf89c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:35.437354 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873aefff-af53-44cf-8499-faf51048b359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:35.438945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873aefff-af53-44cf-8499-faf51048b359 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:35.446170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 445/899] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Jul 25 12:27:35 2022] GET /identity/v3/services => generated 1268 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-873aefff-af53-44cf-8499-faf51048b359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:35.463617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:35.464574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:35.465534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:35.465534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:35.465534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:35.465534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:35.469794 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 455/900] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:35.471269 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3d654-3bd9-4b7e-a8d0-ec17988bf89c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:35.497310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:35.593756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3d654-3bd9-4b7e-a8d0-ec17988bf89c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:35.632955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f573a-9155-4e0c-8a01-2d47db3caa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:35.633863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f573a-9155-4e0c-8a01-2d47db3caa6d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.634335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f573a-9155-4e0c-8a01-2d47db3caa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.635297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f573a-9155-4e0c-8a01-2d47db3caa6d None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.636181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f573a-9155-4e0c-8a01-2d47db3caa6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:35.636483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f573a-9155-4e0c-8a01-2d47db3caa6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:35.637841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f573a-9155-4e0c-8a01-2d47db3caa6d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:35.649672 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 446/901] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:35.655658 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873aefff-af53-44cf-8499-faf51048b359 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:35.667549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a5f14-961e-44f8-8317-bc4cc035f137 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.667857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a5f14-961e-44f8-8317-bc4cc035f137 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.668107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a5f14-961e-44f8-8317-bc4cc035f137 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.669709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 447/902] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:35.674494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96226113-fb63-4307-b721-2320396651cb None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.674781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96226113-fb63-4307-b721-2320396651cb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.675055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96226113-fb63-4307-b721-2320396651cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.693840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-96226113-fb63-4307-b721-2320396651cb None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:35.695042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-96226113-fb63-4307-b721-2320396651cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:35.753373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:35.765428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 448/903] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:35.773504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:35.778497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873aefff-af53-44cf-8499-faf51048b359 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:35.788530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96226113-fb63-4307-b721-2320396651cb None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:35.816089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe915a0c-507f-4578-9ac6-4d0f099771ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:35.817481 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe915a0c-507f-4578-9ac6-4d0f099771ed None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.818021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe915a0c-507f-4578-9ac6-4d0f099771ed None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.818505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe915a0c-507f-4578-9ac6-4d0f099771ed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.827864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe915a0c-507f-4578-9ac6-4d0f099771ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:35.829618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe915a0c-507f-4578-9ac6-4d0f099771ed None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-fe915a0c-507f-4578-9ac6-4d0f099771ed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:35.850537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:35.852275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:35.853474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:35.853474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:35.853474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:35.853474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:35.853474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:35.853474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:35.853474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:35.853474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:35.853474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:27:35.853474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:35.860463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 456/904] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Jul 25 12:27:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:35.860463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bded48-9392-4bbf-801c-5256956451e8 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.860463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bded48-9392-4bbf-801c-5256956451e8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.860463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bded48-9392-4bbf-801c-5256956451e8 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.861664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 457/905] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:35.862404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bded48-9392-4bbf-801c-5256956451e8 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:35.894699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96226113-fb63-4307-b721-2320396651cb None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:35.932453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79d12f-3c9f-4009-b59a-7b4077511311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:35.933123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79d12f-3c9f-4009-b59a-7b4077511311 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.933405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79d12f-3c9f-4009-b59a-7b4077511311 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.933677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79d12f-3c9f-4009-b59a-7b4077511311 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.934451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79d12f-3c9f-4009-b59a-7b4077511311 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:35.935871 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79d12f-3c9f-4009-b59a-7b4077511311 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:35.953983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 449/906] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:27:35 2022] POST /identity/v3/services => generated 241 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:27:35.953983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c092e92-a0ec-4be1-b0fd-05a053e8ad70 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:35.953983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c092e92-a0ec-4be1-b0fd-05a053e8ad70 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:35.953983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c092e92-a0ec-4be1-b0fd-05a053e8ad70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:35.982574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-8c092e92-a0ec-4be1-b0fd-05a053e8ad70 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:35.984170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8c092e92-a0ec-4be1-b0fd-05a053e8ad70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:35.995561 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bded48-9392-4bbf-801c-5256956451e8 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:36.038568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15f10e-d305-41e2-9b98-9dbbbcd0c5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:36.039616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15f10e-d305-41e2-9b98-9dbbbcd0c5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:36.040050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15f10e-d305-41e2-9b98-9dbbbcd0c5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:36.040600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15f10e-d305-41e2-9b98-9dbbbcd0c5a2 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:36.042137 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15f10e-d305-41e2-9b98-9dbbbcd0c5a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:36.042549 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15f10e-d305-41e2-9b98-9dbbbcd0c5a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:36.044358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15f10e-d305-41e2-9b98-9dbbbcd0c5a2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:36.044941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:36.068231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 450/907] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:36.072464 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 458/908] 10.222.0.60 () {56 vars in 1149 bytes} [Mon Jul 25 12:27:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:36.078347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c092e92-a0ec-4be1-b0fd-05a053e8ad70 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:36.090945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15f10e-d305-41e2-9b98-9dbbbcd0c5a2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:36.107963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:36.210206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15f10e-d305-41e2-9b98-9dbbbcd0c5a2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:36.227276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c092e92-a0ec-4be1-b0fd-05a053e8ad70 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:36.253208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7918e0d-0d72-4630-8e2f-97873a64ec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:36.254439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7918e0d-0d72-4630-8e2f-97873a64ec79 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:36.255221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7918e0d-0d72-4630-8e2f-97873a64ec79 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:36.255930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7918e0d-0d72-4630-8e2f-97873a64ec79 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:36.256755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7918e0d-0d72-4630-8e2f-97873a64ec79 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:36.259029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7918e0d-0d72-4630-8e2f-97873a64ec79 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:36.267223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe7fd7-f081-43b0-ad8f-154bbe97a3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:36.268096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe7fd7-f081-43b0-ad8f-154bbe97a3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:36.268527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe7fd7-f081-43b0-ad8f-154bbe97a3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:36.268946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe7fd7-f081-43b0-ad8f-154bbe97a3d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a7918e0d-0d72-4630-8e2f-97873a64ec79 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:36.273326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:36.274589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:36.276119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 459/909] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:27:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:36.281826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7918e0d-0d72-4630-8e2f-97873a64ec79 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:36.336220 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:36.357089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe7fd7-f081-43b0-ad8f-154bbe97a3d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:36.358890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe7fd7-f081-43b0-ad8f-154bbe97a3d8 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:36.370169 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:36.396397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7918e0d-0d72-4630-8e2f-97873a64ec79 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:36.435814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4866813-cc02-4dc5-ad7c-5a04989d3fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:36.436479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4866813-cc02-4dc5-ad7c-5a04989d3fcf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:36.436763 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4866813-cc02-4dc5-ad7c-5a04989d3fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:36.437036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4866813-cc02-4dc5-ad7c-5a04989d3fcf None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:36.437746 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4866813-cc02-4dc5-ad7c-5a04989d3fcf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:36.438083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4866813-cc02-4dc5-ad7c-5a04989d3fcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:36.439665 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4866813-cc02-4dc5-ad7c-5a04989d3fcf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:36.449842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 460/910] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:27:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:36.450714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4866813-cc02-4dc5-ad7c-5a04989d3fcf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:36.470711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 451/911] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:36 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:27:36.472460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe7fd7-f081-43b0-ad8f-154bbe97a3d8 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:36.558182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4866813-cc02-4dc5-ad7c-5a04989d3fcf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:36.563342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:36.583104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe7fd7-f081-43b0-ad8f-154bbe97a3d8 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:36.594999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567b05a-aede-4b47-8235-b561768a9e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:36.595668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567b05a-aede-4b47-8235-b561768a9e72 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:36.596184 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567b05a-aede-4b47-8235-b561768a9e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:36.596184 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567b05a-aede-4b47-8235-b561768a9e72 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:36.596930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567b05a-aede-4b47-8235-b561768a9e72 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:36.597422 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567b05a-aede-4b47-8235-b561768a9e72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:36.598811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567b05a-aede-4b47-8235-b561768a9e72 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:36.604314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 461/912] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Jul 25 12:27:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:36.610143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567b05a-aede-4b47-8235-b561768a9e72 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:36.614206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:36.628714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e0bed-bc61-4f8e-be15-ec79dd021a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:36.630216 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e0bed-bc61-4f8e-be15-ec79dd021a03 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:36.630921 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e0bed-bc61-4f8e-be15-ec79dd021a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:36.631352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e0bed-bc61-4f8e-be15-ec79dd021a03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:36.648608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e0bed-bc61-4f8e-be15-ec79dd021a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:36.650805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e0bed-bc61-4f8e-be15-ec79dd021a03 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-c02e0bed-bc61-4f8e-be15-ec79dd021a03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:36.672219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:36.673190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:36.674123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:36.674123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:36.674123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:36.674123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:36.678317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 452/913] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:36.679334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e0bed-bc61-4f8e-be15-ec79dd021a03 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:36.741123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567b05a-aede-4b47-8235-b561768a9e72 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:36.777133 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e4a5f-a3fa-4e21-a6b5-fe3a439cc41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:36.777943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e4a5f-a3fa-4e21-a6b5-fe3a439cc41c None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:36.778304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e4a5f-a3fa-4e21-a6b5-fe3a439cc41c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:36.778687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e4a5f-a3fa-4e21-a6b5-fe3a439cc41c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:36.779501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e4a5f-a3fa-4e21-a6b5-fe3a439cc41c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:36.781004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e4a5f-a3fa-4e21-a6b5-fe3a439cc41c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:36.802207 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e0bed-bc61-4f8e-be15-ec79dd021a03 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:36.806067 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 462/914] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:27:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:27:36.846844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:36.854424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0d4da-ac0b-490d-b79f-3d76999ccfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:36.855445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0d4da-ac0b-490d-b79f-3d76999ccfdf None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:36.856097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0d4da-ac0b-490d-b79f-3d76999ccfdf None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:36.856881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0d4da-ac0b-490d-b79f-3d76999ccfdf None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:36.857827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0d4da-ac0b-490d-b79f-3d76999ccfdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:36.858332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0d4da-ac0b-490d-b79f-3d76999ccfdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:36.859929 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0d4da-ac0b-490d-b79f-3d76999ccfdf None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:36.873714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 453/915] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:36.883232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e4a5f-a3fa-4e21-a6b5-fe3a439cc41c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:36.954291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:37.001710 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e4a5f-a3fa-4e21-a6b5-fe3a439cc41c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:37.037090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693085d-7175-42ff-bc70-af7cdf86dd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:37.037966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693085d-7175-42ff-bc70-af7cdf86dd1d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:37.038252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693085d-7175-42ff-bc70-af7cdf86dd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:37.038531 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693085d-7175-42ff-bc70-af7cdf86dd1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:37.046127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693085d-7175-42ff-bc70-af7cdf86dd1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:37.047579 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693085d-7175-42ff-bc70-af7cdf86dd1d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-d693085d-7175-42ff-bc70-af7cdf86dd1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:37.063037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:37.063990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:37.065334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:37.065334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:37.065334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:37.065334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:37.065334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:37.065334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:37.065334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:37.065334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:37.065334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 463/916] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:37.070343 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0d4da-ac0b-490d-b79f-3d76999ccfdf None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:37.177661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0d4da-ac0b-490d-b79f-3d76999ccfdf None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:37.219621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359a3ac-851e-4f70-aedf-0a8f4ed9dc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:37.220841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359a3ac-851e-4f70-aedf-0a8f4ed9dc69 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:37.221322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359a3ac-851e-4f70-aedf-0a8f4ed9dc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:37.221792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359a3ac-851e-4f70-aedf-0a8f4ed9dc69 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:37.222839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359a3ac-851e-4f70-aedf-0a8f4ed9dc69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:37.223344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359a3ac-851e-4f70-aedf-0a8f4ed9dc69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:37.224989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359a3ac-851e-4f70-aedf-0a8f4ed9dc69 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:37.240204 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 454/917] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:37.246572 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693085d-7175-42ff-bc70-af7cdf86dd1d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:37.348960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693085d-7175-42ff-bc70-af7cdf86dd1d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:37.370317 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:37.385363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b3c0b-c448-4e62-99e5-4a566e21b802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:37.386074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b3c0b-c448-4e62-99e5-4a566e21b802 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:37.386338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b3c0b-c448-4e62-99e5-4a566e21b802 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:37.386610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b3c0b-c448-4e62-99e5-4a566e21b802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:37.387440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b3c0b-c448-4e62-99e5-4a566e21b802 None admin] RBAC: Adding query filter params (role.id=a1464dd9ad2e4913b9d8cc6ba3c34207, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=6428e810f10c434ba32d073a542b1de7) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:37.387753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b3c0b-c448-4e62-99e5-4a566e21b802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:37.389194 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b3c0b-c448-4e62-99e5-4a566e21b802 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:37.395430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 464/918] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:37 2022] GET /identity/v3/role_assignments?user.id=6428e810f10c434ba32d073a542b1de7&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=a1464dd9ad2e4913b9d8cc6ba3c34207 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:37.562960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819e8fa-e669-4009-85fb-e13eb4960bfa None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:37.563469 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819e8fa-e669-4009-85fb-e13eb4960bfa None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:37.563547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819e8fa-e669-4009-85fb-e13eb4960bfa None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:37.564995 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 455/919] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:37.570536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3d24a-1ce0-477c-ae07-9ff0b31e4f76 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:37.571369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3d24a-1ce0-477c-ae07-9ff0b31e4f76 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:37.571897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3d24a-1ce0-477c-ae07-9ff0b31e4f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:37.590500 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-96b3d24a-1ce0-477c-ae07-9ff0b31e4f76 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:37.592036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-96b3d24a-1ce0-477c-ae07-9ff0b31e4f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:37.660859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 465/920] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:37.684740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819e8fa-e669-4009-85fb-e13eb4960bfa None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:37.739261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:37.793377 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819e8fa-e669-4009-85fb-e13eb4960bfa None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:37.827495 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475ea31-9f4e-40b8-93f0-e3bf1e51613a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:37.828216 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475ea31-9f4e-40b8-93f0-e3bf1e51613a None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:37.828492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475ea31-9f4e-40b8-93f0-e3bf1e51613a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:37.828761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475ea31-9f4e-40b8-93f0-e3bf1e51613a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:37.829408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475ea31-9f4e-40b8-93f0-e3bf1e51613a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:37.830785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475ea31-9f4e-40b8-93f0-e3bf1e51613a None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-d475ea31-9f4e-40b8-93f0-e3bf1e51613a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:37.843425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:37.844351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:37.845177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 456/921] 10.222.0.60 () {56 vars in 1132 bytes} [Mon Jul 25 12:27:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:37.852681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3d24a-1ce0-477c-ae07-9ff0b31e4f76 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:37.958267 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3d24a-1ce0-477c-ae07-9ff0b31e4f76 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:37.973441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:37.994941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343353aa-6462-4a7a-808d-d0226870519e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:37.995610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343353aa-6462-4a7a-808d-d0226870519e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:37.995891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343353aa-6462-4a7a-808d-d0226870519e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:37.996165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343353aa-6462-4a7a-808d-d0226870519e None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:37.996881 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343353aa-6462-4a7a-808d-d0226870519e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:37.997188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343353aa-6462-4a7a-808d-d0226870519e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:37.998884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343353aa-6462-4a7a-808d-d0226870519e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:38.004073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 466/922] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Jul 25 12:27:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:38.009794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475ea31-9f4e-40b8-93f0-e3bf1e51613a None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:38.119690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475ea31-9f4e-40b8-93f0-e3bf1e51613a None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:38.127119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7357d-b977-48f0-811a-4ba40a37db90 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:38.127690 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7357d-b977-48f0-811a-4ba40a37db90 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:38.128169 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7357d-b977-48f0-811a-4ba40a37db90 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:38.130151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 467/923] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:38.134073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:38.136891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ee728-de71-4912-b479-23bdc36f4962 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:38.137397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ee728-de71-4912-b479-23bdc36f4962 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:38.137873 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ee728-de71-4912-b479-23bdc36f4962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:38.155344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d8662-20a7-4f79-9e73-c6e290574013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:38.156003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d8662-20a7-4f79-9e73-c6e290574013 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:38.156288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d8662-20a7-4f79-9e73-c6e290574013 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:38.156563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d8662-20a7-4f79-9e73-c6e290574013 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:38.157270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d8662-20a7-4f79-9e73-c6e290574013 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:38.157574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d8662-20a7-4f79-9e73-c6e290574013 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:38.159004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-173ee728-de71-4912-b479-23bdc36f4962 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:38.159130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d8662-20a7-4f79-9e73-c6e290574013 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:38.160630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-173ee728-de71-4912-b479-23bdc36f4962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:38.169290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 457/924] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Jul 25 12:27:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:38.170836 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d8662-20a7-4f79-9e73-c6e290574013 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:38.234594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 468/925] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:38.257433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ee728-de71-4912-b479-23bdc36f4962 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:38.282292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d8662-20a7-4f79-9e73-c6e290574013 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:38.319273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a9fac-aa61-49bf-b50a-5e7f90ca6b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:38.319932 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a9fac-aa61-49bf-b50a-5e7f90ca6b95 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:38.320213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a9fac-aa61-49bf-b50a-5e7f90ca6b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:38.320489 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a9fac-aa61-49bf-b50a-5e7f90ca6b95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:38.321242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a9fac-aa61-49bf-b50a-5e7f90ca6b95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:38.322654 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a9fac-aa61-49bf-b50a-5e7f90ca6b95 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:38.344227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 458/926] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:27:38 2022] POST /identity/v3/endpoints => generated 347 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:27:38.374914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ee728-de71-4912-b479-23bdc36f4962 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:38.386102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:38.420655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55260ea6-e0ab-4e8c-baa8-d6790a5efd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:38.421469 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55260ea6-e0ab-4e8c-baa8-d6790a5efd64 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:38.421746 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55260ea6-e0ab-4e8c-baa8-d6790a5efd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:38.422040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55260ea6-e0ab-4e8c-baa8-d6790a5efd64 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:38.422691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55260ea6-e0ab-4e8c-baa8-d6790a5efd64 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:38.424077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55260ea6-e0ab-4e8c-baa8-d6790a5efd64 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-55260ea6-e0ab-4e8c-baa8-d6790a5efd64 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:38.437856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:38.438966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:38.439816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 469/927] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Jul 25 12:27:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:38.447347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a9fac-aa61-49bf-b50a-5e7f90ca6b95 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:38.486127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:38.563985 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a9fac-aa61-49bf-b50a-5e7f90ca6b95 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:38.604026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318e395-d00f-4edb-b5d6-fda2b673ed45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:38.605096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318e395-d00f-4edb-b5d6-fda2b673ed45 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:38.605378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318e395-d00f-4edb-b5d6-fda2b673ed45 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:38.605651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318e395-d00f-4edb-b5d6-fda2b673ed45 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:38.606691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318e395-d00f-4edb-b5d6-fda2b673ed45 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:38.606996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318e395-d00f-4edb-b5d6-fda2b673ed45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:38.608419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318e395-d00f-4edb-b5d6-fda2b673ed45 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:38.613645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 459/928] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Jul 25 12:27:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:38.619752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55260ea6-e0ab-4e8c-baa8-d6790a5efd64 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:38.734216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55260ea6-e0ab-4e8c-baa8-d6790a5efd64 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:38.769563 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa4e72-f417-402a-a099-28ab53f9179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:38.770618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa4e72-f417-402a-a099-28ab53f9179f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:38.771125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa4e72-f417-402a-a099-28ab53f9179f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:38.771602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa4e72-f417-402a-a099-28ab53f9179f None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:38.772592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa4e72-f417-402a-a099-28ab53f9179f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:38.773108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa4e72-f417-402a-a099-28ab53f9179f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:38.774702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa4e72-f417-402a-a099-28ab53f9179f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:38.780920 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 470/929] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Jul 25 12:27:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:38.786986 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318e395-d00f-4edb-b5d6-fda2b673ed45 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:38.897467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:38.903835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318e395-d00f-4edb-b5d6-fda2b673ed45 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:38.939896 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59800793-c250-49e5-88ed-788656e8d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:38.940903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59800793-c250-49e5-88ed-788656e8d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:38.941410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59800793-c250-49e5-88ed-788656e8d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:38.941881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59800793-c250-49e5-88ed-788656e8d7e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:38.942949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59800793-c250-49e5-88ed-788656e8d7e7 None admin] RBAC: Adding query filter params (interface=public, service_id=476a528767fb458f8b1e7ed35aebd4f8, region_id=RegionOne) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:38.943462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59800793-c250-49e5-88ed-788656e8d7e7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:38.945107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59800793-c250-49e5-88ed-788656e8d7e7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:38.951407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 460/930] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Jul 25 12:27:38 2022] GET /identity/v3/endpoints?service_id=476a528767fb458f8b1e7ed35aebd4f8&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:38.957124 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa4e72-f417-402a-a099-28ab53f9179f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:38.965120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b41a3b-151f-4a61-819f-d2241896d67c None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:38.965406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b41a3b-151f-4a61-819f-d2241896d67c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:38.965684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b41a3b-151f-4a61-819f-d2241896d67c None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:38.968291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 461/931] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:38.972292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef53bf8-e46e-4b8a-b07b-01e9f4ad765e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:38.972584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef53bf8-e46e-4b8a-b07b-01e9f4ad765e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:38.972867 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef53bf8-e46e-4b8a-b07b-01e9f4ad765e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:38.991802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-bef53bf8-e46e-4b8a-b07b-01e9f4ad765e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:38.992972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-bef53bf8-e46e-4b8a-b07b-01e9f4ad765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:38.995000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:39.070032 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 462/932] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:39.074629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa4e72-f417-402a-a099-28ab53f9179f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:39.084445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef53bf8-e46e-4b8a-b07b-01e9f4ad765e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:39.113856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b449ce-2876-4825-8015-d3c10030a030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:39.115101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b449ce-2876-4825-8015-d3c10030a030 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:39.115595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b449ce-2876-4825-8015-d3c10030a030 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:39.116068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b449ce-2876-4825-8015-d3c10030a030 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:39.116962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b449ce-2876-4825-8015-d3c10030a030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:39.119375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b449ce-2876-4825-8015-d3c10030a030 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:39.126387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 471/933] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Jul 25 12:27:38 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:27:39.191101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:39.203304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef53bf8-e46e-4b8a-b07b-01e9f4ad765e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:39.245245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1500e0-2bf7-4c44-bb0b-4e094a7f6f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:39.246399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1500e0-2bf7-4c44-bb0b-4e094a7f6f2d None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:39.246938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1500e0-2bf7-4c44-bb0b-4e094a7f6f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:39.247413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1500e0-2bf7-4c44-bb0b-4e094a7f6f2d None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:39.248338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1500e0-2bf7-4c44-bb0b-4e094a7f6f2d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:39.250034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1500e0-2bf7-4c44-bb0b-4e094a7f6f2d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:39.253082 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: WARNING keystone.server.flask.common [None req-2d1500e0-2bf7-4c44-bb0b-4e094a7f6f2d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 12:27:39.273557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.sql.core [None req-2d1500e0-2bf7-4c44-bb0b-4e094a7f6f2d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 12:27:39.273557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 12:27:39.273557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [parameters: {'id': '628164e6d8834c82ac2ab9ceab0a5998', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 12:27:39.273557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80326) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-2d1500e0-2bf7-4c44-bb0b-4e094a7f6f2d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 628164e6d8834c82ac2ab9ceab0a5998.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 628164e6d8834c82ac2ab9ceab0a5998. Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application conn.query(q) Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 12:27:39.293203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application result.read() Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 25 12:27:39.294143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application next(self.gen) Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application yield resource Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application next(self.gen) Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise self.value Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application session.commit() Jul 25 12:27:39.295075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self.session.flush() Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application compat.raise_( Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 12:27:39.295955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 12:27:39.297554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application conn.query(q) Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application result.read() Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 12:27:39.298451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [parameters: {'id': '628164e6d8834c82ac2ab9ceab0a5998', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:39.299363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:39.300139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:39.302260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:39.302260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 25 12:27:39.302260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 25 12:27:39.302260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:39.302260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:39.302260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 25 12:27:39.302260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 25 12:27:39.302260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 628164e6d8834c82ac2ab9ceab0a5998. Jul 25 12:27:39.302260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:39.305270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 463/934] 10.222.0.60 () {60 vars in 1140 bytes} [Mon Jul 25 12:27:39 2022] POST /identity/v3/projects => generated 262 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:39.311219 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b449ce-2876-4825-8015-d3c10030a030 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:39.410639 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:39.417452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b449ce-2876-4825-8015-d3c10030a030 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:39.453864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb0dc3-addc-4829-885e-5e297cacb3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:39.454688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb0dc3-addc-4829-885e-5e297cacb3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:39.454972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb0dc3-addc-4829-885e-5e297cacb3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:39.455249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb0dc3-addc-4829-885e-5e297cacb3b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:39.462165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb0dc3-addc-4829-885e-5e297cacb3b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:39.463646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb0dc3-addc-4829-885e-5e297cacb3b9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-f1eb0dc3-addc-4829-885e-5e297cacb3b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:39.477498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:39.478480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:39.479633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:39.479633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:39.479633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:39.479633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:39.479633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:39.479633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:39.479633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:39.479633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:39.479633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 472/935] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:39.486937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1500e0-2bf7-4c44-bb0b-4e094a7f6f2d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:39.523077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219d978-9c42-4724-a757-4c342a723503 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:39.523355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219d978-9c42-4724-a757-4c342a723503 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:39.523555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219d978-9c42-4724-a757-4c342a723503 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:39.525192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 473/936] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:39.530016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ab446-f0da-43cc-9f8e-908b2e7e28fd None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:39.530302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ab446-f0da-43cc-9f8e-908b2e7e28fd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:39.530579 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ab446-f0da-43cc-9f8e-908b2e7e28fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:39.551354 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-939ab446-f0da-43cc-9f8e-908b2e7e28fd None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:39.552537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-939ab446-f0da-43cc-9f8e-908b2e7e28fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:39.618568 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1500e0-2bf7-4c44-bb0b-4e094a7f6f2d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:39.634265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 474/937] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:39.653937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ab446-f0da-43cc-9f8e-908b2e7e28fd None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:39.657163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7d7bf-956a-4a89-8edc-6491b99b5468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:39.661951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7d7bf-956a-4a89-8edc-6491b99b5468 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:39.661951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7d7bf-956a-4a89-8edc-6491b99b5468 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:39.661951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7d7bf-956a-4a89-8edc-6491b99b5468 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:39.661951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7d7bf-956a-4a89-8edc-6491b99b5468 None admin] RBAC: Adding query filter params (name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:39.661951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7d7bf-956a-4a89-8edc-6491b99b5468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:39.661951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7d7bf-956a-4a89-8edc-6491b99b5468 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:39.693975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 464/938] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:39.760344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:39.777435 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ab446-f0da-43cc-9f8e-908b2e7e28fd None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:39.813748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196d2b9-cca5-46e9-a185-0676d0ef1dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:39.814613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196d2b9-cca5-46e9-a185-0676d0ef1dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:39.814894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196d2b9-cca5-46e9-a185-0676d0ef1dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:39.815168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196d2b9-cca5-46e9-a185-0676d0ef1dd6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:39.827475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196d2b9-cca5-46e9-a185-0676d0ef1dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-5196d2b9-cca5-46e9-a185-0676d0ef1dd6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:39.835871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:39.836912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:39.838821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 475/939] 10.222.0.60 () {56 vars in 1120 bytes} [Mon Jul 25 12:27:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:39.844057 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7d7bf-956a-4a89-8edc-6491b99b5468 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:39.855301 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:39.947540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7d7bf-956a-4a89-8edc-6491b99b5468 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:39.982252 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889067d-a09d-4b3c-9424-73de0d59fe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:39.982927 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889067d-a09d-4b3c-9424-73de0d59fe04 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:39.983205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889067d-a09d-4b3c-9424-73de0d59fe04 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:39.983475 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889067d-a09d-4b3c-9424-73de0d59fe04 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:39.984222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889067d-a09d-4b3c-9424-73de0d59fe04 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:39.984520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889067d-a09d-4b3c-9424-73de0d59fe04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:39.985883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889067d-a09d-4b3c-9424-73de0d59fe04 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:39.994975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 465/940] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Jul 25 12:27:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:40.000585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196d2b9-cca5-46e9-a185-0676d0ef1dd6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:40.104013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196d2b9-cca5-46e9-a185-0676d0ef1dd6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:40.139352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99791506-4038-4d8b-aa62-0ec2872469b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:40.140143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99791506-4038-4d8b-aa62-0ec2872469b8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:40.140420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99791506-4038-4d8b-aa62-0ec2872469b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:40.140690 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99791506-4038-4d8b-aa62-0ec2872469b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:40.151695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99791506-4038-4d8b-aa62-0ec2872469b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:40.153295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99791506-4038-4d8b-aa62-0ec2872469b8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:40.160068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-99791506-4038-4d8b-aa62-0ec2872469b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:40.171296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:40.172216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:40.173520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:40.173520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:40.173520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:40.173520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:40.173520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 476/941] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:40.178485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889067d-a09d-4b3c-9424-73de0d59fe04 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:40.261224 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:40.291862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889067d-a09d-4b3c-9424-73de0d59fe04 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:40.329209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554bd1c-adc4-4c73-b0e1-832147a39a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:40.329879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554bd1c-adc4-4c73-b0e1-832147a39a9b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:40.330194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554bd1c-adc4-4c73-b0e1-832147a39a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:40.330475 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554bd1c-adc4-4c73-b0e1-832147a39a9b None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:40.331200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554bd1c-adc4-4c73-b0e1-832147a39a9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:40.331507 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554bd1c-adc4-4c73-b0e1-832147a39a9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:40.332874 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554bd1c-adc4-4c73-b0e1-832147a39a9b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:40.345222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 466/942] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:40.350548 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99791506-4038-4d8b-aa62-0ec2872469b8 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:40.456866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99791506-4038-4d8b-aa62-0ec2872469b8 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:40.491517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9aaf7c-dd01-4db4-9d7b-7945d3dd90d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:40.492326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9aaf7c-dd01-4db4-9d7b-7945d3dd90d8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:40.492604 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9aaf7c-dd01-4db4-9d7b-7945d3dd90d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:40.492874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9aaf7c-dd01-4db4-9d7b-7945d3dd90d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:40.501330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9aaf7c-dd01-4db4-9d7b-7945d3dd90d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:40.502796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9aaf7c-dd01-4db4-9d7b-7945d3dd90d8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-bf9aaf7c-dd01-4db4-9d7b-7945d3dd90d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:40.517323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:40.518311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:40.519501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:40.519501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:40.519501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:40.519501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:40.519501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:40.519501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:40.519501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:40.519501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:40.519501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:27:40.519501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:40.519501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 477/943] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Jul 25 12:27:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:40.524570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554bd1c-adc4-4c73-b0e1-832147a39a9b None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:40.554728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38f397-6f84-48d5-8dbe-f61881bb6444 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:40.555009 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38f397-6f84-48d5-8dbe-f61881bb6444 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:40.555284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38f397-6f84-48d5-8dbe-f61881bb6444 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:40.557495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 478/944] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:40.562735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c230691-e400-4673-a3c3-62f8572b5c4e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:40.563127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c230691-e400-4673-a3c3-62f8572b5c4e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:40.563551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c230691-e400-4673-a3c3-62f8572b5c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:40.583034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9c230691-e400-4673-a3c3-62f8572b5c4e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:40.584791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9c230691-e400-4673-a3c3-62f8572b5c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:40.626626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554bd1c-adc4-4c73-b0e1-832147a39a9b None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:40.654743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:40.662320 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc34c8-f6f9-4f9a-b36d-18731a9f31c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:40.663168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc34c8-f6f9-4f9a-b36d-18731a9f31c1 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:40.663531 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc34c8-f6f9-4f9a-b36d-18731a9f31c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:40.663914 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc34c8-f6f9-4f9a-b36d-18731a9f31c1 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:40.664913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc34c8-f6f9-4f9a-b36d-18731a9f31c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:40.665289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc34c8-f6f9-4f9a-b36d-18731a9f31c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:40.667476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc34c8-f6f9-4f9a-b36d-18731a9f31c1 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:40.674222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:40.678288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 479/945] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:40.699007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c230691-e400-4673-a3c3-62f8572b5c4e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:40.710159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 467/946] 10.222.0.60 () {56 vars in 1149 bytes} [Mon Jul 25 12:27:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:40.716373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc34c8-f6f9-4f9a-b36d-18731a9f31c1 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:40.816641 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c230691-e400-4673-a3c3-62f8572b5c4e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:40.831567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc34c8-f6f9-4f9a-b36d-18731a9f31c1 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:40.860481 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b4132-198e-45f7-8003-4debd042fcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:40.861776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b4132-198e-45f7-8003-4debd042fcbd None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:40.862605 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b4132-198e-45f7-8003-4debd042fcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:40.862882 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b4132-198e-45f7-8003-4debd042fcbd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:40.863864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b4132-198e-45f7-8003-4debd042fcbd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:40.865754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b4132-198e-45f7-8003-4debd042fcbd None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:40.879780 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443db6e0-7719-4a0b-a4b5-1ea443500fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:40.880601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443db6e0-7719-4a0b-a4b5-1ea443500fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:40.880879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443db6e0-7719-4a0b-a4b5-1ea443500fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:40.881150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443db6e0-7719-4a0b-a4b5-1ea443500fb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-031b4132-198e-45f7-8003-4debd042fcbd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:40.884183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:40.885252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:27:40.886129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:40.888957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 480/947] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:40.894186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443db6e0-7719-4a0b-a4b5-1ea443500fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:40.894186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443db6e0-7719-4a0b-a4b5-1ea443500fb3 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:40.898932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b4132-198e-45f7-8003-4debd042fcbd None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-443db6e0-7719-4a0b-a4b5-1ea443500fb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:40.916846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:40.917825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:40.918776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:40.918776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:40.918776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:40.918776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:40.921989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 468/948] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:40.924874 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443db6e0-7719-4a0b-a4b5-1ea443500fb3 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:40.967341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:40.979445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:41.023533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b4132-198e-45f7-8003-4debd042fcbd None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:41.043647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443db6e0-7719-4a0b-a4b5-1ea443500fb3 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:41.061445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e81732-6485-414c-9589-9ec30f686ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:41.062550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e81732-6485-414c-9589-9ec30f686ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.063241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e81732-6485-414c-9589-9ec30f686ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.063722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e81732-6485-414c-9589-9ec30f686ca9 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.064646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e81732-6485-414c-9589-9ec30f686ca9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:41.065202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e81732-6485-414c-9589-9ec30f686ca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:41.066801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e81732-6485-414c-9589-9ec30f686ca9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:41.077366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 481/949] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:41.078569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e81732-6485-414c-9589-9ec30f686ca9 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:41.079132 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19120e2b-5308-4210-846d-2272621d585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:41.079861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19120e2b-5308-4210-846d-2272621d585a None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.080141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19120e2b-5308-4210-846d-2272621d585a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.080416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19120e2b-5308-4210-846d-2272621d585a None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.081141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19120e2b-5308-4210-846d-2272621d585a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:41.081442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19120e2b-5308-4210-846d-2272621d585a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:41.082930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19120e2b-5308-4210-846d-2272621d585a None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:41.095635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 469/950] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:41.101367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19120e2b-5308-4210-846d-2272621d585a None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:41.201063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e81732-6485-414c-9589-9ec30f686ca9 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:41.205509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:41.208287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19120e2b-5308-4210-846d-2272621d585a None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:41.229430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:41.249834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e1f43-603d-4fb5-9ecb-56dc09a6c735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:41.250939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e1f43-603d-4fb5-9ecb-56dc09a6c735 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.251378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e1f43-603d-4fb5-9ecb-56dc09a6c735 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.251832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e1f43-603d-4fb5-9ecb-56dc09a6c735 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.252806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e1f43-603d-4fb5-9ecb-56dc09a6c735 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:41.253068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13bf0c-765c-4a6e-8426-f8b89e1f709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:41.253854 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e1f43-603d-4fb5-9ecb-56dc09a6c735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:41.254077 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13bf0c-765c-4a6e-8426-f8b89e1f709d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.254426 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13bf0c-765c-4a6e-8426-f8b89e1f709d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.255166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13bf0c-765c-4a6e-8426-f8b89e1f709d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.256272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e1f43-603d-4fb5-9ecb-56dc09a6c735 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:41.267369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13bf0c-765c-4a6e-8426-f8b89e1f709d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:41.267369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13bf0c-765c-4a6e-8426-f8b89e1f709d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:41.268268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 482/951] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:41 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-ca13bf0c-765c-4a6e-8426-f8b89e1f709d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:41.282715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:41.283752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:41.284704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:41.284704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:41.284704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:41.284704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:41.284704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:41.284704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:41.284704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:41.284704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:41.284704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 470/952] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:41.290344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e1f43-603d-4fb5-9ecb-56dc09a6c735 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:41.319546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb56d7-6009-4c4b-96bb-61cc5ec40190 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.319885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb56d7-6009-4c4b-96bb-61cc5ec40190 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.320095 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb56d7-6009-4c4b-96bb-61cc5ec40190 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.321633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 471/953] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:41.327881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b140a14-5b01-4b2b-a46b-8a3a3da74cd5 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.328170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b140a14-5b01-4b2b-a46b-8a3a3da74cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.328454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b140a14-5b01-4b2b-a46b-8a3a3da74cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.349321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-1b140a14-5b01-4b2b-a46b-8a3a3da74cd5 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:41.350948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-1b140a14-5b01-4b2b-a46b-8a3a3da74cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:41.415637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e1f43-603d-4fb5-9ecb-56dc09a6c735 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:41.426620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 472/954] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:41.450290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b140a14-5b01-4b2b-a46b-8a3a3da74cd5 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:41.466276 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47658359-2a5d-4b6f-afc1-9acd659496b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:41.467093 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47658359-2a5d-4b6f-afc1-9acd659496b1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.467389 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47658359-2a5d-4b6f-afc1-9acd659496b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.467665 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47658359-2a5d-4b6f-afc1-9acd659496b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.468460 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47658359-2a5d-4b6f-afc1-9acd659496b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:41.468768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47658359-2a5d-4b6f-afc1-9acd659496b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:41.470265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47658359-2a5d-4b6f-afc1-9acd659496b1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:41.487266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 483/955] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:41.495034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47658359-2a5d-4b6f-afc1-9acd659496b1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:41.548830 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:41.550581 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:41.562484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b140a14-5b01-4b2b-a46b-8a3a3da74cd5 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:41.598386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81808d7a-1fd8-4d46-9f89-7192c786f2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:41.599196 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81808d7a-1fd8-4d46-9f89-7192c786f2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.599484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81808d7a-1fd8-4d46-9f89-7192c786f2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.599757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81808d7a-1fd8-4d46-9f89-7192c786f2e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.600424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81808d7a-1fd8-4d46-9f89-7192c786f2e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:41.601800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81808d7a-1fd8-4d46-9f89-7192c786f2e5 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-81808d7a-1fd8-4d46-9f89-7192c786f2e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:41.614358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:41.615460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:41.616429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:41.620697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47658359-2a5d-4b6f-afc1-9acd659496b1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:41.621240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 473/956] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Jul 25 12:27:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:41.622506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81808d7a-1fd8-4d46-9f89-7192c786f2e5 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:41.658229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4d649-c0a9-48a9-8f29-934cada5aca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:41.659864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4d649-c0a9-48a9-8f29-934cada5aca9 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.660351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4d649-c0a9-48a9-8f29-934cada5aca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.660828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4d649-c0a9-48a9-8f29-934cada5aca9 None admin] PATH_INFO: `/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/6428e810f10c434ba32d073a542b1de7/roles/a1464dd9ad2e4913b9d8cc6ba3c34207` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.663298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4d649-c0a9-48a9-8f29-934cada5aca9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff5a41f8ca6438295c512792d2022e4, user_id=6428e810f10c434ba32d073a542b1de7, role_id=a1464dd9ad2e4913b9d8cc6ba3c34207)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:41.664985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4d649-c0a9-48a9-8f29-934cada5aca9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:41.685990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 484/957] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:27:41 2022] PUT /identity/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/6428e810f10c434ba32d073a542b1de7/roles/a1464dd9ad2e4913b9d8cc6ba3c34207 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:27:41.744898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81808d7a-1fd8-4d46-9f89-7192c786f2e5 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:41.787569 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5365fe-f614-4b31-aef1-3ff1b0f78eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:41.788546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5365fe-f614-4b31-aef1-3ff1b0f78eae None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.788994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5365fe-f614-4b31-aef1-3ff1b0f78eae None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.789429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5365fe-f614-4b31-aef1-3ff1b0f78eae None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.790352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5365fe-f614-4b31-aef1-3ff1b0f78eae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:41.790864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5365fe-f614-4b31-aef1-3ff1b0f78eae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:41.792508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5365fe-f614-4b31-aef1-3ff1b0f78eae None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:41.802499 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 474/958] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Jul 25 12:27:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:41.805212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4d649-c0a9-48a9-8f29-934cada5aca9 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:41.824105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4770492-75ba-4a92-bff5-803baff5fcb1 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.824659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4770492-75ba-4a92-bff5-803baff5fcb1 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.825155 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4770492-75ba-4a92-bff5-803baff5fcb1 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.827076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 475/959] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:41.836079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed158d7e-38e0-4185-b2a1-8c5b8fd25ea4 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.836612 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed158d7e-38e0-4185-b2a1-8c5b8fd25ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.837040 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed158d7e-38e0-4185-b2a1-8c5b8fd25ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.859331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-ed158d7e-38e0-4185-b2a1-8c5b8fd25ea4 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:41.862014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ed158d7e-38e0-4185-b2a1-8c5b8fd25ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:41.881300 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:41.923621 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:41.927437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4d649-c0a9-48a9-8f29-934cada5aca9 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:41.957021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 476/960] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:41.969973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f280e93-56de-4c8e-b5b5-a003cadc6772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:41.969973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f280e93-56de-4c8e-b5b5-a003cadc6772 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:41.969973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f280e93-56de-4c8e-b5b5-a003cadc6772 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:41.969973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f280e93-56de-4c8e-b5b5-a003cadc6772 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:41.970383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f280e93-56de-4c8e-b5b5-a003cadc6772 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:41.970383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f280e93-56de-4c8e-b5b5-a003cadc6772 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:41.970383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f280e93-56de-4c8e-b5b5-a003cadc6772 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:41.975022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 485/961] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Jul 25 12:27:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:41.981039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed158d7e-38e0-4185-b2a1-8c5b8fd25ea4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:41.985779 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f280e93-56de-4c8e-b5b5-a003cadc6772 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:42.097823 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed158d7e-38e0-4185-b2a1-8c5b8fd25ea4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:42.110243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f280e93-56de-4c8e-b5b5-a003cadc6772 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:42.143654 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd8c6e-088a-4211-94ad-2b73efd262ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:42.144308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd8c6e-088a-4211-94ad-2b73efd262ef None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:42.144583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd8c6e-088a-4211-94ad-2b73efd262ef None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:42.144853 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd8c6e-088a-4211-94ad-2b73efd262ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:42.145594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd8c6e-088a-4211-94ad-2b73efd262ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:42.146977 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd8c6e-088a-4211-94ad-2b73efd262ef None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:42.155245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a5bbb-cdd2-48dc-92f0-ab8a746df12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:42.156889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a5bbb-cdd2-48dc-92f0-ab8a746df12a None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:42.157660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a5bbb-cdd2-48dc-92f0-ab8a746df12a None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:42.157895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a5bbb-cdd2-48dc-92f0-ab8a746df12a None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:42.158895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a5bbb-cdd2-48dc-92f0-ab8a746df12a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:42.160576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a5bbb-cdd2-48dc-92f0-ab8a746df12a None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:42.164616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: WARNING keystone.server.flask.common [None req-8d2a5bbb-cdd2-48dc-92f0-ab8a746df12a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 12:27:42.170151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 477/962] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:27:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:27:42.192113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.sql.core [None req-8d2a5bbb-cdd2-48dc-92f0-ab8a746df12a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 12:27:42.192113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 12:27:42.192113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [parameters: {'user_id': '84194d6a9a7745b69ef2dfbc133d9165', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 12:27:42.192113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80327) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-8d2a5bbb-cdd2-48dc-92f0-ab8a746df12a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application conn.query(q) Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result.read() Jul 25 12:27:42.201053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 12:27:42.202251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application next(self.gen) Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application yield resource Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application next(self.gen) Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise self.value Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application session.commit() Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 12:27:42.203453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.session.flush() Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application compat.raise_( Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 12:27:42.204392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 12:27:42.205313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application conn.query(q) Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application result.read() Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 12:27:42.206285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [parameters: {'user_id': '84194d6a9a7745b69ef2dfbc133d9165', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:42.207266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 25 12:27:42.208285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 25 12:27:42.209258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 25 12:27:42.209258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 25 12:27:42.209258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 12:27:42.209258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 12:27:42.209258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 12:27:42.209258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:42.211655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 486/963] 10.222.0.60 () {60 vars in 1131 bytes} [Mon Jul 25 12:27:41 2022] POST /identity/v3/users => generated 160 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:42.217840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd8c6e-088a-4211-94ad-2b73efd262ef None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:42.223243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:42.349450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd8c6e-088a-4211-94ad-2b73efd262ef None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:42.385181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e96bf8-40a3-4c67-9033-1ce17936fda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:42.386006 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e96bf8-40a3-4c67-9033-1ce17936fda0 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:42.386288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e96bf8-40a3-4c67-9033-1ce17936fda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:42.386560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e96bf8-40a3-4c67-9033-1ce17936fda0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:42.394797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e96bf8-40a3-4c67-9033-1ce17936fda0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:42.396228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e96bf8-40a3-4c67-9033-1ce17936fda0 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-84e96bf8-40a3-4c67-9033-1ce17936fda0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:42.411434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:42.412406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:42.413758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:42.413758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:42.413758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:42.413758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:42.413758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:42.413758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:42.413758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:42.413758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:42.413758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:27:42.413758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:42.413758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 478/964] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:42.419459 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a5bbb-cdd2-48dc-92f0-ab8a746df12a None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:42.448854 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:42.527848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a5bbb-cdd2-48dc-92f0-ab8a746df12a None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:42.563328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16285d9-d1f0-42c1-bacc-2a990600f595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:42.564004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16285d9-d1f0-42c1-bacc-2a990600f595 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:42.564281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16285d9-d1f0-42c1-bacc-2a990600f595 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:42.564553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16285d9-d1f0-42c1-bacc-2a990600f595 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:42.565479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16285d9-d1f0-42c1-bacc-2a990600f595 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:42.565676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16285d9-d1f0-42c1-bacc-2a990600f595 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:42.567501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16285d9-d1f0-42c1-bacc-2a990600f595 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:42.612857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 487/965] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Jul 25 12:27:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:43.390572 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44089b05-5639-4e6e-8472-590bf3a70389 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:43.391101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44089b05-5639-4e6e-8472-590bf3a70389 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:43.391101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44089b05-5639-4e6e-8472-590bf3a70389 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:43.392634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 479/966] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:43.397705 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245aa2e-4865-42c2-9c4e-294e02dfd8be None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:43.398118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245aa2e-4865-42c2-9c4e-294e02dfd8be None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:43.398283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245aa2e-4865-42c2-9c4e-294e02dfd8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:43.417224 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1245aa2e-4865-42c2-9c4e-294e02dfd8be None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:43.418777 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1245aa2e-4865-42c2-9c4e-294e02dfd8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:43.486513 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 488/967] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:43.509025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44089b05-5639-4e6e-8472-590bf3a70389 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:43.573927 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:43.616603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44089b05-5639-4e6e-8472-590bf3a70389 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:43.649311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053de06c-fe0f-431f-bc23-4413bcfdd3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:43.650086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053de06c-fe0f-431f-bc23-4413bcfdd3d0 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:43.650371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053de06c-fe0f-431f-bc23-4413bcfdd3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:43.650643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053de06c-fe0f-431f-bc23-4413bcfdd3d0 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:43.651380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053de06c-fe0f-431f-bc23-4413bcfdd3d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:43.652743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053de06c-fe0f-431f-bc23-4413bcfdd3d0 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:43.669239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 480/968] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:27:43 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:27:43.802797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da524a5e-a869-40ef-b855-6b1533da6f40 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:43.803180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da524a5e-a869-40ef-b855-6b1533da6f40 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:43.803391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da524a5e-a869-40ef-b855-6b1533da6f40 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:43.804942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 489/969] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:43.810800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd6eee-1941-4e34-b17d-8a158f08a6d4 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:43.811086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd6eee-1941-4e34-b17d-8a158f08a6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:43.811371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd6eee-1941-4e34-b17d-8a158f08a6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:43.831871 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-73bd6eee-1941-4e34-b17d-8a158f08a6d4 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:43.832331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-73bd6eee-1941-4e34-b17d-8a158f08a6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:43.909160 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 481/970] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:43.932974 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da524a5e-a869-40ef-b855-6b1533da6f40 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:43.948689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:44.036653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da524a5e-a869-40ef-b855-6b1533da6f40 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:44.070787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c829746-9a37-4f65-b17f-bd4a362dfbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:44.071526 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c829746-9a37-4f65-b17f-bd4a362dfbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:44.071840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c829746-9a37-4f65-b17f-bd4a362dfbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:44.072113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c829746-9a37-4f65-b17f-bd4a362dfbc8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:44.082185 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c829746-9a37-4f65-b17f-bd4a362dfbc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-4c829746-9a37-4f65-b17f-bd4a362dfbc8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:44.090848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:44.091821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:44.092702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 490/971] 10.222.0.60 () {56 vars in 1120 bytes} [Mon Jul 25 12:27:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:44.099527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd6eee-1941-4e34-b17d-8a158f08a6d4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:44.147709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:44.211310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd6eee-1941-4e34-b17d-8a158f08a6d4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:44.250499 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa80db1-6511-4d0e-8eb4-98cea84dc198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:44.251174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa80db1-6511-4d0e-8eb4-98cea84dc198 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:44.251458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa80db1-6511-4d0e-8eb4-98cea84dc198 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:44.251735 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa80db1-6511-4d0e-8eb4-98cea84dc198 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:44.252493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa80db1-6511-4d0e-8eb4-98cea84dc198 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:44.252802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa80db1-6511-4d0e-8eb4-98cea84dc198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:44.254193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa80db1-6511-4d0e-8eb4-98cea84dc198 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:44.263056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 482/972] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Jul 25 12:27:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:44.264129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7d296-1993-40ff-bbb5-da826164d0ba None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:44.264423 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7d296-1993-40ff-bbb5-da826164d0ba None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:44.264728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7d296-1993-40ff-bbb5-da826164d0ba None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:44.266329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 491/973] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:44.268365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa80db1-6511-4d0e-8eb4-98cea84dc198 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:44.272196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c406c-be3f-4828-b1d0-50d95073e912 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:44.272485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c406c-be3f-4828-b1d0-50d95073e912 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:44.272768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c406c-be3f-4828-b1d0-50d95073e912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:44.291589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f86c406c-be3f-4828-b1d0-50d95073e912 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:44.293182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f86c406c-be3f-4828-b1d0-50d95073e912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:44.362157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 492/974] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:44.375447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa80db1-6511-4d0e-8eb4-98cea84dc198 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:44.383653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c406c-be3f-4828-b1d0-50d95073e912 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:44.385388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:44.388544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:44.420487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988d3aa-d710-4efb-981f-24eec528ab42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:44.421285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988d3aa-d710-4efb-981f-24eec528ab42 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:44.421571 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988d3aa-d710-4efb-981f-24eec528ab42 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:44.421850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988d3aa-d710-4efb-981f-24eec528ab42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:44.435614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988d3aa-d710-4efb-981f-24eec528ab42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:44.437500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988d3aa-d710-4efb-981f-24eec528ab42 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-4988d3aa-d710-4efb-981f-24eec528ab42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:44.459922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:44.460811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:44.461699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:44.461699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:44.461699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:44.461699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:44.463185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 483/975] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:44.467536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988d3aa-d710-4efb-981f-24eec528ab42 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:44.487514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c406c-be3f-4828-b1d0-50d95073e912 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:44.523980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2745ca-3a66-4cfe-a883-043a1a0755a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:44.524812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2745ca-3a66-4cfe-a883-043a1a0755a2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:44.525099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2745ca-3a66-4cfe-a883-043a1a0755a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:44.525380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2745ca-3a66-4cfe-a883-043a1a0755a2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:44.526062 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2745ca-3a66-4cfe-a883-043a1a0755a2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:44.527436 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2745ca-3a66-4cfe-a883-043a1a0755a2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-2d2745ca-3a66-4cfe-a883-043a1a0755a2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:44.539984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:44.541076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:44.547232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 493/976] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Jul 25 12:27:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:44.548079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2745ca-3a66-4cfe-a883-043a1a0755a2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:44.575608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988d3aa-d710-4efb-981f-24eec528ab42 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:44.610417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc282dd3-381f-4072-a6a1-2863db452150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:44.611069 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc282dd3-381f-4072-a6a1-2863db452150 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:44.611380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc282dd3-381f-4072-a6a1-2863db452150 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:44.611647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc282dd3-381f-4072-a6a1-2863db452150 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:44.612413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc282dd3-381f-4072-a6a1-2863db452150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:44.612659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc282dd3-381f-4072-a6a1-2863db452150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:44.614033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc282dd3-381f-4072-a6a1-2863db452150 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:44.616978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:44.627041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 484/977] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:44.632126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc282dd3-381f-4072-a6a1-2863db452150 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:44.654764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:44.660197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2745ca-3a66-4cfe-a883-043a1a0755a2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:44.698176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c212cf-bccf-4c19-8d5c-47e8287935e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:44.698935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c212cf-bccf-4c19-8d5c-47e8287935e8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:44.699299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c212cf-bccf-4c19-8d5c-47e8287935e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:44.699649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c212cf-bccf-4c19-8d5c-47e8287935e8 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:44.700437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c212cf-bccf-4c19-8d5c-47e8287935e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:44.700819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c212cf-bccf-4c19-8d5c-47e8287935e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:44.702226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c212cf-bccf-4c19-8d5c-47e8287935e8 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:44.707822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 494/978] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Jul 25 12:27:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:44.713443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c212cf-bccf-4c19-8d5c-47e8287935e8 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:44.733562 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc282dd3-381f-4072-a6a1-2863db452150 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:44.772341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe295a0-8f08-4fb2-b7d2-8b563765c7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:44.773156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe295a0-8f08-4fb2-b7d2-8b563765c7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:44.773442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe295a0-8f08-4fb2-b7d2-8b563765c7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:44.773720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe295a0-8f08-4fb2-b7d2-8b563765c7d6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:44.781888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe295a0-8f08-4fb2-b7d2-8b563765c7d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:44.783442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe295a0-8f08-4fb2-b7d2-8b563765c7d6 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-5fe295a0-8f08-4fb2-b7d2-8b563765c7d6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:44.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:44.801012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:44.803027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:44.803027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:44.803027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:44.803027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:44.803027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:44.803027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:44.803027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:44.803027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:44.803027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:27:44.803027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:44.803027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 485/979] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Jul 25 12:27:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:44.807276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe295a0-8f08-4fb2-b7d2-8b563765c7d6 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:44.828126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c212cf-bccf-4c19-8d5c-47e8287935e8 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:44.860157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:44.865824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9f6ff-500d-4c7b-81f4-68363050d990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:44.866461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9f6ff-500d-4c7b-81f4-68363050d990 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:44.866741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9f6ff-500d-4c7b-81f4-68363050d990 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:44.867013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9f6ff-500d-4c7b-81f4-68363050d990 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:44.867721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9f6ff-500d-4c7b-81f4-68363050d990 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:44.868024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9f6ff-500d-4c7b-81f4-68363050d990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:44.869944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9f6ff-500d-4c7b-81f4-68363050d990 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:44.875564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 495/980] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Jul 25 12:27:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:44.881388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9f6ff-500d-4c7b-81f4-68363050d990 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:44.901924 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:44.925112 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe295a0-8f08-4fb2-b7d2-8b563765c7d6 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:44.964961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91f9bb-2f49-43dd-b6a3-3896863de9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:44.965725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91f9bb-2f49-43dd-b6a3-3896863de9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:44.965994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91f9bb-2f49-43dd-b6a3-3896863de9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:44.966220 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91f9bb-2f49-43dd-b6a3-3896863de9b7 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:44.967354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91f9bb-2f49-43dd-b6a3-3896863de9b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:44.967800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91f9bb-2f49-43dd-b6a3-3896863de9b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:44.969091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91f9bb-2f49-43dd-b6a3-3896863de9b7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:44.987666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 486/981] 10.222.0.60 () {56 vars in 1149 bytes} [Mon Jul 25 12:27:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:44.992960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91f9bb-2f49-43dd-b6a3-3896863de9b7 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:44.994186 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9f6ff-500d-4c7b-81f4-68363050d990 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:45.030211 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e367a1-6111-4b6e-b639-600f173bfd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:45.030877 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e367a1-6111-4b6e-b639-600f173bfd7e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:45.031163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e367a1-6111-4b6e-b639-600f173bfd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:45.031442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e367a1-6111-4b6e-b639-600f173bfd7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:45.032240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e367a1-6111-4b6e-b639-600f173bfd7e None admin] RBAC: Adding query filter params (interface=admin, service_id=476a528767fb458f8b1e7ed35aebd4f8, region_id=RegionOne) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:45.032554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e367a1-6111-4b6e-b639-600f173bfd7e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:45.034359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e367a1-6111-4b6e-b639-600f173bfd7e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:45.040287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 496/982] 10.222.0.60 () {56 vars in 1249 bytes} [Mon Jul 25 12:27:44 2022] GET /identity/v3/endpoints?service_id=476a528767fb458f8b1e7ed35aebd4f8&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:45.046076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e367a1-6111-4b6e-b639-600f173bfd7e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:45.096996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91f9bb-2f49-43dd-b6a3-3896863de9b7 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:45.108373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:45.134087 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f7cd9-9153-4d3b-9678-5a60f7f275ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:45.134699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f7cd9-9153-4d3b-9678-5a60f7f275ea None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:45.134984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f7cd9-9153-4d3b-9678-5a60f7f275ea None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:45.135260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f7cd9-9153-4d3b-9678-5a60f7f275ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:45.144414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:45.154209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e367a1-6111-4b6e-b639-600f173bfd7e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:45.193972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d7492-76e7-4e8c-ab3c-5c36cc804c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:45.194647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d7492-76e7-4e8c-ab3c-5c36cc804c7d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:45.194952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d7492-76e7-4e8c-ab3c-5c36cc804c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:45.195194 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d7492-76e7-4e8c-ab3c-5c36cc804c7d None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:45.195867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d7492-76e7-4e8c-ab3c-5c36cc804c7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:45.197265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d7492-76e7-4e8c-ab3c-5c36cc804c7d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:45.203797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 497/983] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Jul 25 12:27:45 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:27:45.216597 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f7cd9-9153-4d3b-9678-5a60f7f275ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:45.218120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f7cd9-9153-4d3b-9678-5a60f7f275ea None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:45.333999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 487/984] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:44 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:27:45.335209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d7492-76e7-4e8c-ab3c-5c36cc804c7d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:45.439660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d7492-76e7-4e8c-ab3c-5c36cc804c7d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:45.463112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:45.475306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0538d3b-4448-4939-9b55-732618b5d6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:45.476103 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0538d3b-4448-4939-9b55-732618b5d6fa None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:45.476391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0538d3b-4448-4939-9b55-732618b5d6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:45.476670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0538d3b-4448-4939-9b55-732618b5d6fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:45.488122 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0538d3b-4448-4939-9b55-732618b5d6fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:45.489580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0538d3b-4448-4939-9b55-732618b5d6fa None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-f0538d3b-4448-4939-9b55-732618b5d6fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:45.506585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:45.507517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:45.508489 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:45.508489 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:45.508489 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:45.508489 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:45.508489 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 498/985] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:45.514567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f7cd9-9153-4d3b-9678-5a60f7f275ea None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:45.555226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:45.616209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f7cd9-9153-4d3b-9678-5a60f7f275ea None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:45.650652 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccc1e5-c51d-4fe9-924a-b453ff45c61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:45.651319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccc1e5-c51d-4fe9-924a-b453ff45c61b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:45.651666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccc1e5-c51d-4fe9-924a-b453ff45c61b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:45.651876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccc1e5-c51d-4fe9-924a-b453ff45c61b None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:45.652603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccc1e5-c51d-4fe9-924a-b453ff45c61b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:45.652913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccc1e5-c51d-4fe9-924a-b453ff45c61b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:45.654297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccc1e5-c51d-4fe9-924a-b453ff45c61b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:45.667150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 488/986] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:45.672910 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0538d3b-4448-4939-9b55-732618b5d6fa None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:45.680466 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe4c77-1ec7-4f12-9379-7cbc6854c67d None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:45.680761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe4c77-1ec7-4f12-9379-7cbc6854c67d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:45.681045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe4c77-1ec7-4f12-9379-7cbc6854c67d None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:45.682582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 489/987] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:45.687332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095677ac-a999-4af2-98eb-9f46b4e52afb None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:45.687619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095677ac-a999-4af2-98eb-9f46b4e52afb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:45.687898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095677ac-a999-4af2-98eb-9f46b4e52afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:45.705991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-095677ac-a999-4af2-98eb-9f46b4e52afb None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:45.707127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-095677ac-a999-4af2-98eb-9f46b4e52afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:45.777944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 490/988] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:45.781642 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0538d3b-4448-4939-9b55-732618b5d6fa None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:45.799219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095677ac-a999-4af2-98eb-9f46b4e52afb None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:45.818976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24079ccf-bb21-4fe6-a643-39d419af3d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:45.819777 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24079ccf-bb21-4fe6-a643-39d419af3d13 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:45.820093 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24079ccf-bb21-4fe6-a643-39d419af3d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:45.820346 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24079ccf-bb21-4fe6-a643-39d419af3d13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:45.827265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24079ccf-bb21-4fe6-a643-39d419af3d13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:45.827414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:45.828811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24079ccf-bb21-4fe6-a643-39d419af3d13 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-24079ccf-bb21-4fe6-a643-39d419af3d13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:45.843376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:45.844386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:45.845404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:45.845404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:45.845404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:45.845404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:45.845404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:45.845404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:45.845404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:45.845404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:45.845404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 499/989] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:45.850228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24079ccf-bb21-4fe6-a643-39d419af3d13 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:45.889369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:45.908302 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095677ac-a999-4af2-98eb-9f46b4e52afb None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:45.944120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327f8a8-9dd9-42dd-8262-665f12ae39dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:45.944923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327f8a8-9dd9-42dd-8262-665f12ae39dd None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:45.945207 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327f8a8-9dd9-42dd-8262-665f12ae39dd None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:45.945485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327f8a8-9dd9-42dd-8262-665f12ae39dd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:45.946183 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327f8a8-9dd9-42dd-8262-665f12ae39dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:45.948092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327f8a8-9dd9-42dd-8262-665f12ae39dd None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-5327f8a8-9dd9-42dd-8262-665f12ae39dd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:45.960053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:45.961031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:45.962096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 491/990] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:45.967717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327f8a8-9dd9-42dd-8262-665f12ae39dd None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:45.968240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24079ccf-bb21-4fe6-a643-39d419af3d13 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:46.004251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5fef3-0029-4705-b2d5-8e25bf7fa5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:46.005031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5fef3-0029-4705-b2d5-8e25bf7fa5bb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:46.005382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5fef3-0029-4705-b2d5-8e25bf7fa5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:46.005768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5fef3-0029-4705-b2d5-8e25bf7fa5bb None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:46.006603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5fef3-0029-4705-b2d5-8e25bf7fa5bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:46.006915 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5fef3-0029-4705-b2d5-8e25bf7fa5bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:46.008484 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5fef3-0029-4705-b2d5-8e25bf7fa5bb None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:46.021419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 500/991] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:46.027408 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5fef3-0029-4705-b2d5-8e25bf7fa5bb None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:46.076725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:46.087715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327f8a8-9dd9-42dd-8262-665f12ae39dd None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:46.122410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57caf2c3-b02b-4719-828c-48ce1c8b2a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:46.123070 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57caf2c3-b02b-4719-828c-48ce1c8b2a63 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:46.123355 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57caf2c3-b02b-4719-828c-48ce1c8b2a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:46.123635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57caf2c3-b02b-4719-828c-48ce1c8b2a63 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:46.124339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57caf2c3-b02b-4719-828c-48ce1c8b2a63 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:46.124652 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57caf2c3-b02b-4719-828c-48ce1c8b2a63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:46.126041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57caf2c3-b02b-4719-828c-48ce1c8b2a63 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:46.130479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:46.131885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 492/992] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:46.138082 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57caf2c3-b02b-4719-828c-48ce1c8b2a63 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:46.140586 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5fef3-0029-4705-b2d5-8e25bf7fa5bb None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:46.175443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5f670-feb5-4a64-9095-8d3f22f95a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:46.176108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5f670-feb5-4a64-9095-8d3f22f95a46 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:46.176393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5f670-feb5-4a64-9095-8d3f22f95a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:46.176668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5f670-feb5-4a64-9095-8d3f22f95a46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:46.177504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5f670-feb5-4a64-9095-8d3f22f95a46 None admin] RBAC: Adding query filter params (role.id=a1464dd9ad2e4913b9d8cc6ba3c34207, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=6428e810f10c434ba32d073a542b1de7) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:46.177819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5f670-feb5-4a64-9095-8d3f22f95a46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:46.179295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5f670-feb5-4a64-9095-8d3f22f95a46 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:46.193959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 501/993] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:46 2022] GET /identity/v3/role_assignments?user.id=6428e810f10c434ba32d073a542b1de7&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=a1464dd9ad2e4913b9d8cc6ba3c34207 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:46.260226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57caf2c3-b02b-4719-828c-48ce1c8b2a63 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:46.297180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a5063-852a-4ef2-ae7d-66400628fbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:46.297862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a5063-852a-4ef2-ae7d-66400628fbfd None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:46.298170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a5063-852a-4ef2-ae7d-66400628fbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:46.298443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a5063-852a-4ef2-ae7d-66400628fbfd None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:46.299174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a5063-852a-4ef2-ae7d-66400628fbfd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:46.299477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a5063-852a-4ef2-ae7d-66400628fbfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:46.300902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a5063-852a-4ef2-ae7d-66400628fbfd None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:46.312578 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 493/994] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:46.314367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5f670-feb5-4a64-9095-8d3f22f95a46 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:46.424156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5f670-feb5-4a64-9095-8d3f22f95a46 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:46.458070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c091fc-ba51-4630-8343-28034f69dba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:46.458728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c091fc-ba51-4630-8343-28034f69dba0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:46.459009 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c091fc-ba51-4630-8343-28034f69dba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:46.459283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c091fc-ba51-4630-8343-28034f69dba0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:46.460072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c091fc-ba51-4630-8343-28034f69dba0 None admin] RBAC: Adding query filter params (interface=public, service_id=ca11221dee584199b3ec9c62b72f11cd, region_id=RegionOne) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:46.460384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c091fc-ba51-4630-8343-28034f69dba0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:46.461741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c091fc-ba51-4630-8343-28034f69dba0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:46.467083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 502/995] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Jul 25 12:27:46 2022] GET /identity/v3/endpoints?service_id=ca11221dee584199b3ec9c62b72f11cd&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:46.472748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a5063-852a-4ef2-ae7d-66400628fbfd None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:46.497528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:46.574784 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a5063-852a-4ef2-ae7d-66400628fbfd None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:46.607938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b248c-1fa1-4fb9-a79f-a1546f2cc5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:46.608593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b248c-1fa1-4fb9-a79f-a1546f2cc5ca None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:46.608870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b248c-1fa1-4fb9-a79f-a1546f2cc5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:46.609138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b248c-1fa1-4fb9-a79f-a1546f2cc5ca None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:46.609806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b248c-1fa1-4fb9-a79f-a1546f2cc5ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:46.611182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b248c-1fa1-4fb9-a79f-a1546f2cc5ca None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:46.617545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 494/996] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Jul 25 12:27:46 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:27:47.216216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42400367-adaa-4553-ae12-8597b52987d9 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:47.216689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42400367-adaa-4553-ae12-8597b52987d9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:47.216764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42400367-adaa-4553-ae12-8597b52987d9 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:47.218332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 503/997] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:47.223325 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5419af7-d60f-458c-b76c-e14d29b2c080 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:47.223619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5419af7-d60f-458c-b76c-e14d29b2c080 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:47.223896 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5419af7-d60f-458c-b76c-e14d29b2c080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:47.242793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-c5419af7-d60f-458c-b76c-e14d29b2c080 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:47.244532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c5419af7-d60f-458c-b76c-e14d29b2c080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:47.311059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 495/998] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:47.333132 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42400367-adaa-4553-ae12-8597b52987d9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:47.360286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:47.437980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42400367-adaa-4553-ae12-8597b52987d9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:47.470607 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11198241-e695-435b-b17f-2d8cd9948b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:47.471405 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11198241-e695-435b-b17f-2d8cd9948b9c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:47.471688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11198241-e695-435b-b17f-2d8cd9948b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:47.471964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11198241-e695-435b-b17f-2d8cd9948b9c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:47.472611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11198241-e695-435b-b17f-2d8cd9948b9c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:47.474009 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11198241-e695-435b-b17f-2d8cd9948b9c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-11198241-e695-435b-b17f-2d8cd9948b9c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:47.486733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:47.487664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:27:47.488506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:47.489881 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 504/999] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Jul 25 12:27:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:47.494704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5419af7-d60f-458c-b76c-e14d29b2c080 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:47.509508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:47.592599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5419af7-d60f-458c-b76c-e14d29b2c080 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:47.625881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d4d55-1b07-4172-ae1f-57637cc246e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:47.626575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d4d55-1b07-4172-ae1f-57637cc246e3 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:47.626860 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d4d55-1b07-4172-ae1f-57637cc246e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:47.627138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d4d55-1b07-4172-ae1f-57637cc246e3 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:47.627847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d4d55-1b07-4172-ae1f-57637cc246e3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:47.628154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d4d55-1b07-4172-ae1f-57637cc246e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:47.629512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d4d55-1b07-4172-ae1f-57637cc246e3 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:47.634470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 496/1000] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Jul 25 12:27:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:47.639933 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11198241-e695-435b-b17f-2d8cd9948b9c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:47.731743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:47.745310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11198241-e695-435b-b17f-2d8cd9948b9c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:47.778541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d065894-4ac7-4bae-b15c-58baad396e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:47.779206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d065894-4ac7-4bae-b15c-58baad396e35 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:47.779485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d065894-4ac7-4bae-b15c-58baad396e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:47.779756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d065894-4ac7-4bae-b15c-58baad396e35 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:47.780458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d065894-4ac7-4bae-b15c-58baad396e35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:47.780760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d065894-4ac7-4bae-b15c-58baad396e35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:47.782143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d065894-4ac7-4bae-b15c-58baad396e35 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:47.787207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 505/1001] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Jul 25 12:27:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:47.793310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d4d55-1b07-4172-ae1f-57637cc246e3 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:47.870103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:47.892907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d4d55-1b07-4172-ae1f-57637cc246e3 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:47.927217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd8f3e-5555-4b40-b4e3-15d8fdc39a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:47.927927 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd8f3e-5555-4b40-b4e3-15d8fdc39a91 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:47.928137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd8f3e-5555-4b40-b4e3-15d8fdc39a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:47.928453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd8f3e-5555-4b40-b4e3-15d8fdc39a91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:47.929304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd8f3e-5555-4b40-b4e3-15d8fdc39a91 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:47.930680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd8f3e-5555-4b40-b4e3-15d8fdc39a91 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:47.951601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 497/1002] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:27:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:27:48.239938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c14638-6cc8-4c3f-ace8-8eca094aeda6 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:48.240441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c14638-6cc8-4c3f-ace8-8eca094aeda6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:48.240519 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c14638-6cc8-4c3f-ace8-8eca094aeda6 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:48.241883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 506/1003] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:48.247623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3553476-9682-4cc2-b849-ba4786e0bf6d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:48.247916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3553476-9682-4cc2-b849-ba4786e0bf6d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:48.248192 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3553476-9682-4cc2-b849-ba4786e0bf6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:48.266853 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-d3553476-9682-4cc2-b849-ba4786e0bf6d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:48.268152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d3553476-9682-4cc2-b849-ba4786e0bf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:48.341464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 498/1004] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:48 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:48.363188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c14638-6cc8-4c3f-ace8-8eca094aeda6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:48.467093 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c14638-6cc8-4c3f-ace8-8eca094aeda6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:48.500876 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58456a16-eaa9-4c22-aafc-b5194f762d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:48.501667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58456a16-eaa9-4c22-aafc-b5194f762d2b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:48.501974 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58456a16-eaa9-4c22-aafc-b5194f762d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:48.502253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58456a16-eaa9-4c22-aafc-b5194f762d2b None admin] PATH_INFO: `/v3/services/image` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:48.502909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58456a16-eaa9-4c22-aafc-b5194f762d2b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:48.504275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58456a16-eaa9-4c22-aafc-b5194f762d2b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-58456a16-eaa9-4c22-aafc-b5194f762d2b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:48.517648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:48.518824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:27:48.519742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:48.522245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 507/1005] 10.222.0.60 () {56 vars in 1105 bytes} [Mon Jul 25 12:27:48 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:48.525455 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3553476-9682-4cc2-b849-ba4786e0bf6d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:48.586569 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:48.607075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cb740-ee4a-4745-85a0-d3010546a7a6 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:48.607359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cb740-ee4a-4745-85a0-d3010546a7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:48.607636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cb740-ee4a-4745-85a0-d3010546a7a6 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:48.609153 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 508/1006] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:48.613654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661317f-3fa7-4442-84d1-d8f8cd6f2ee0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:48.613958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661317f-3fa7-4442-84d1-d8f8cd6f2ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:48.614247 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661317f-3fa7-4442-84d1-d8f8cd6f2ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:48.630179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3553476-9682-4cc2-b849-ba4786e0bf6d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:48.632463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9661317f-3fa7-4442-84d1-d8f8cd6f2ee0 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:48.633645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9661317f-3fa7-4442-84d1-d8f8cd6f2ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:48.645383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:48.669130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54201d85-34f8-4600-a939-28591b0c0cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:48.669797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54201d85-34f8-4600-a939-28591b0c0cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:48.670104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54201d85-34f8-4600-a939-28591b0c0cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:48.670394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54201d85-34f8-4600-a939-28591b0c0cd3 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:48.671086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54201d85-34f8-4600-a939-28591b0c0cd3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:48.672060 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54201d85-34f8-4600-a939-28591b0c0cd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:48.672748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54201d85-34f8-4600-a939-28591b0c0cd3 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:48.677896 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 499/1007] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:48.684026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54201d85-34f8-4600-a939-28591b0c0cd3 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:48.705243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 509/1008] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:48 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:48.727245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661317f-3fa7-4442-84d1-d8f8cd6f2ee0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:48.787121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54201d85-34f8-4600-a939-28591b0c0cd3 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:48.816816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:48.826348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac4d95-582b-4ec1-8748-6f66ccfd4cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:48.826995 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac4d95-582b-4ec1-8748-6f66ccfd4cbb None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:48.827270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac4d95-582b-4ec1-8748-6f66ccfd4cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:48.827542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac4d95-582b-4ec1-8748-6f66ccfd4cbb None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:48.828245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac4d95-582b-4ec1-8748-6f66ccfd4cbb None admin] RBAC: Adding query filter params (type=image) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:48.828550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac4d95-582b-4ec1-8748-6f66ccfd4cbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:48.829944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac4d95-582b-4ec1-8748-6f66ccfd4cbb None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:48.832318 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661317f-3fa7-4442-84d1-d8f8cd6f2ee0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:48.834508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 500/1009] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:48.870619 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ce341-157c-4acc-93f7-10c4d9189ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:48.871446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce341-157c-4acc-93f7-10c4d9189ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:48.871730 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce341-157c-4acc-93f7-10c4d9189ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:48.872007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce341-157c-4acc-93f7-10c4d9189ad4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:48.872670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ce341-157c-4acc-93f7-10c4d9189ad4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:48.874093 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ce341-157c-4acc-93f7-10c4d9189ad4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a60ce341-157c-4acc-93f7-10c4d9189ad4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:48.887324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:48.888447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:48.894899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 510/1010] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:48.895714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac4d95-582b-4ec1-8748-6f66ccfd4cbb None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:49.007270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac4d95-582b-4ec1-8748-6f66ccfd4cbb None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:49.044145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cb762-be53-47d7-a18e-3bed64112aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:49.045229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cb762-be53-47d7-a18e-3bed64112aae None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:49.045737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cb762-be53-47d7-a18e-3bed64112aae None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:49.046255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cb762-be53-47d7-a18e-3bed64112aae None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:49.047167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cb762-be53-47d7-a18e-3bed64112aae None admin] RBAC: Adding query filter params (name=compute) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:49.047704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cb762-be53-47d7-a18e-3bed64112aae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:49.049288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cb762-be53-47d7-a18e-3bed64112aae None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:49.055088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 501/1011] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:49.061077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ce341-157c-4acc-93f7-10c4d9189ad4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:49.074813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:49.175504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ce341-157c-4acc-93f7-10c4d9189ad4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:49.211298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b837d4-78a2-4993-b08b-5ffbefd8aae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:49.211962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b837d4-78a2-4993-b08b-5ffbefd8aae5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:49.212754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b837d4-78a2-4993-b08b-5ffbefd8aae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:49.213032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b837d4-78a2-4993-b08b-5ffbefd8aae5 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:49.213726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b837d4-78a2-4993-b08b-5ffbefd8aae5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:49.214064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b837d4-78a2-4993-b08b-5ffbefd8aae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:49.215430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b837d4-78a2-4993-b08b-5ffbefd8aae5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:49.220545 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 511/1012] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Jul 25 12:27:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:49.226153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cb762-be53-47d7-a18e-3bed64112aae None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:49.284588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:49.329243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cb762-be53-47d7-a18e-3bed64112aae None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:49.363184 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e110fc-6766-4258-81be-b9f199685473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:49.363833 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e110fc-6766-4258-81be-b9f199685473 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:49.364112 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e110fc-6766-4258-81be-b9f199685473 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:49.364384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e110fc-6766-4258-81be-b9f199685473 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:49.365130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e110fc-6766-4258-81be-b9f199685473 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:49.366591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e110fc-6766-4258-81be-b9f199685473 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:49.393939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 502/1013] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:27:49 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:27:49.964173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed6917-7847-4bc5-9753-336da744b622 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:49.964740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed6917-7847-4bc5-9753-336da744b622 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:49.964740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed6917-7847-4bc5-9753-336da744b622 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:49.966257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 512/1014] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:49.971271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5983334-d9c2-423d-8a5f-3231217a049c None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:49.971662 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5983334-d9c2-423d-8a5f-3231217a049c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:49.971839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5983334-d9c2-423d-8a5f-3231217a049c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:49.990481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b5983334-d9c2-423d-8a5f-3231217a049c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:49.991680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b5983334-d9c2-423d-8a5f-3231217a049c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:50.063500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 503/1015] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:49 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:50.085867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed6917-7847-4bc5-9753-336da744b622 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:50.163702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:50.195538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed6917-7847-4bc5-9753-336da744b622 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:50.229566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b341af1-0485-4723-894e-ad3a2b4a4ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:50.230322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b341af1-0485-4723-894e-ad3a2b4a4ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:50.230605 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b341af1-0485-4723-894e-ad3a2b4a4ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:50.230878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b341af1-0485-4723-894e-ad3a2b4a4ad6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:50.232303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b341af1-0485-4723-894e-ad3a2b4a4ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:50.233738 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b341af1-0485-4723-894e-ad3a2b4a4ad6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:50.235625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 513/1016] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:50.241181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5983334-d9c2-423d-8a5f-3231217a049c None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:50.283607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:50.349784 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5983334-d9c2-423d-8a5f-3231217a049c None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:50.384454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e413647-1e27-4afe-8f84-53d67f56b8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:50.385121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e413647-1e27-4afe-8f84-53d67f56b8b5 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:50.385398 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e413647-1e27-4afe-8f84-53d67f56b8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:50.385675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e413647-1e27-4afe-8f84-53d67f56b8b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:50.386421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e413647-1e27-4afe-8f84-53d67f56b8b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:50.387844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e413647-1e27-4afe-8f84-53d67f56b8b5 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:50.413715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 504/1017] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:27:50 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:27:50.849791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902728e-6ed0-4e54-b47e-8e0bb8e06b18 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:50.850171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902728e-6ed0-4e54-b47e-8e0bb8e06b18 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:50.850395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902728e-6ed0-4e54-b47e-8e0bb8e06b18 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:50.851771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 514/1018] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:50.858004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449549bd-04ad-4eb0-ad6b-e29cf84472c5 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:50.858297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449549bd-04ad-4eb0-ad6b-e29cf84472c5 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:50.858574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449549bd-04ad-4eb0-ad6b-e29cf84472c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:50.876765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-449549bd-04ad-4eb0-ad6b-e29cf84472c5 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:50.877969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-449549bd-04ad-4eb0-ad6b-e29cf84472c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:50.949316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 505/1019] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:50 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:50.972381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902728e-6ed0-4e54-b47e-8e0bb8e06b18 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:51.075477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902728e-6ed0-4e54-b47e-8e0bb8e06b18 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:51.109992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cae11c-2e50-4bca-8b82-5d6908267e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:51.110572 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cae11c-2e50-4bca-8b82-5d6908267e3e None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:51.110858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cae11c-2e50-4bca-8b82-5d6908267e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:51.111135 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cae11c-2e50-4bca-8b82-5d6908267e3e None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:51.111867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cae11c-2e50-4bca-8b82-5d6908267e3e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:51.113234 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cae11c-2e50-4bca-8b82-5d6908267e3e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:51.137624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 515/1020] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:27:50 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:27:51.497351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d32c3-aad6-4d79-bd10-f8b15303178f None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:51.497840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d32c3-aad6-4d79-bd10-f8b15303178f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:51.497840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d32c3-aad6-4d79-bd10-f8b15303178f None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:51.499378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 506/1021] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:51.504467 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9e420-2fdc-45e5-a183-2727617ba8de None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:51.504770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9e420-2fdc-45e5-a183-2727617ba8de None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:51.505055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9e420-2fdc-45e5-a183-2727617ba8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:51.512012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:51.525846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-44c9e420-2fdc-45e5-a183-2727617ba8de None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:51.527221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-44c9e420-2fdc-45e5-a183-2727617ba8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:51.605071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 516/1022] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:51.628420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d32c3-aad6-4d79-bd10-f8b15303178f None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:51.651645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:51.734802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d32c3-aad6-4d79-bd10-f8b15303178f None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:51.767948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ef2dc-0600-4a75-98d8-9b04fab9fb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:51.768691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ef2dc-0600-4a75-98d8-9b04fab9fb31 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:51.768973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ef2dc-0600-4a75-98d8-9b04fab9fb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:51.769248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ef2dc-0600-4a75-98d8-9b04fab9fb31 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:51.779316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ef2dc-0600-4a75-98d8-9b04fab9fb31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-354ef2dc-0600-4a75-98d8-9b04fab9fb31 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:51.787393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:51.788280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:51.789124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:51.789124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:51.789124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:51.789124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:51.789124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:51.789124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:51.789124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:51.789124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:51.789124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:51.789124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:51.789124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:27:51.789124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:51.794178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 507/1023] 10.222.0.60 () {56 vars in 1120 bytes} [Mon Jul 25 12:27:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:51.795404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9e420-2fdc-45e5-a183-2727617ba8de None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:51.896150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9e420-2fdc-45e5-a183-2727617ba8de None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:51.922596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:51.932481 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ceb9b8-6ec4-4f04-9e68-046c12c28d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:51.933158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ceb9b8-6ec4-4f04-9e68-046c12c28d41 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:51.933440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ceb9b8-6ec4-4f04-9e68-046c12c28d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:51.933714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ceb9b8-6ec4-4f04-9e68-046c12c28d41 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:51.934518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ceb9b8-6ec4-4f04-9e68-046c12c28d41 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:51.934822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ceb9b8-6ec4-4f04-9e68-046c12c28d41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:51.936200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ceb9b8-6ec4-4f04-9e68-046c12c28d41 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:51.949288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 517/1024] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Jul 25 12:27:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:51.950458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ef2dc-0600-4a75-98d8-9b04fab9fb31 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:52.030284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:52.050308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ef2dc-0600-4a75-98d8-9b04fab9fb31 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:52.082626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d164f-a744-49a6-829b-ff69895a5bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:52.083420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d164f-a744-49a6-829b-ff69895a5bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:52.083702 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d164f-a744-49a6-829b-ff69895a5bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:52.083974 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d164f-a744-49a6-829b-ff69895a5bb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:52.095042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d164f-a744-49a6-829b-ff69895a5bb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:52.096470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d164f-a744-49a6-829b-ff69895a5bb2 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-060d164f-a744-49a6-829b-ff69895a5bb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:52.113976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:52.114890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:52.120797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:52.120797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:52.120797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:52.120797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:52.120797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 508/1025] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:52.121090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ceb9b8-6ec4-4f04-9e68-046c12c28d41 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:52.229819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ceb9b8-6ec4-4f04-9e68-046c12c28d41 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:52.265044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efebd9e-b23b-4016-88e9-6c077b4b534d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:52.266010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efebd9e-b23b-4016-88e9-6c077b4b534d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:52.266436 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efebd9e-b23b-4016-88e9-6c077b4b534d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:52.266855 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efebd9e-b23b-4016-88e9-6c077b4b534d None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:52.267721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efebd9e-b23b-4016-88e9-6c077b4b534d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:52.268190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efebd9e-b23b-4016-88e9-6c077b4b534d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:52.269693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efebd9e-b23b-4016-88e9-6c077b4b534d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:52.284594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 518/1026] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:52.291149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d164f-a744-49a6-829b-ff69895a5bb2 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:52.393866 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d164f-a744-49a6-829b-ff69895a5bb2 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:52.422515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:52.428614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75611f80-bda7-405e-86bc-dcb1cb26bff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:52.429758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75611f80-bda7-405e-86bc-dcb1cb26bff1 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:52.430481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75611f80-bda7-405e-86bc-dcb1cb26bff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:52.430765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75611f80-bda7-405e-86bc-dcb1cb26bff1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:52.438356 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75611f80-bda7-405e-86bc-dcb1cb26bff1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:52.439797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75611f80-bda7-405e-86bc-dcb1cb26bff1 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-75611f80-bda7-405e-86bc-dcb1cb26bff1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:52.454052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:52.455021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:52.455973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:52.455973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:52.455973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:52.455973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:52.455973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:52.455973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:52.455973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:52.455973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:52.455973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:52.455973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:52.458227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 509/1027] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Jul 25 12:27:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:52.460893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efebd9e-b23b-4016-88e9-6c077b4b534d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:52.466237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d8fd8-4c46-498a-ac16-07cf6385e4d4 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:52.466524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d8fd8-4c46-498a-ac16-07cf6385e4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:52.466800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d8fd8-4c46-498a-ac16-07cf6385e4d4 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:52.468279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 510/1028] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:52.472846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd09f5-9b8d-4531-873a-1851f87190cb None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:52.473130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd09f5-9b8d-4531-873a-1851f87190cb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:52.473411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd09f5-9b8d-4531-873a-1851f87190cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:52.491308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-91bd09f5-9b8d-4531-873a-1851f87190cb None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:52.492471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-91bd09f5-9b8d-4531-873a-1851f87190cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:52.522365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:52.561797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 511/1029] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:52.586801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efebd9e-b23b-4016-88e9-6c077b4b534d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:52.588486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd09f5-9b8d-4531-873a-1851f87190cb None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:52.625287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8657-5543-424b-b967-7efff3dd52f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:52.626535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8657-5543-424b-b967-7efff3dd52f6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:52.626535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8657-5543-424b-b967-7efff3dd52f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:52.626884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8657-5543-424b-b967-7efff3dd52f6 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:52.627717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d8657-5543-424b-b967-7efff3dd52f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:52.628022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d8657-5543-424b-b967-7efff3dd52f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:52.629444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d8657-5543-424b-b967-7efff3dd52f6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:52.651802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 519/1030] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Jul 25 12:27:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:52.652817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8657-5543-424b-b967-7efff3dd52f6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:52.707668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd09f5-9b8d-4531-873a-1851f87190cb None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:52.737311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:52.751237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ce66f-69e9-4a21-8e4f-ae677c46fdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:52.752026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ce66f-69e9-4a21-8e4f-ae677c46fdaa None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:52.752304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ce66f-69e9-4a21-8e4f-ae677c46fdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:52.752612 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ce66f-69e9-4a21-8e4f-ae677c46fdaa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:52.753727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ce66f-69e9-4a21-8e4f-ae677c46fdaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:52.755169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ce66f-69e9-4a21-8e4f-ae677c46fdaa None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:52.756939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 512/1031] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:52.758814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8657-5543-424b-b967-7efff3dd52f6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:52.762965 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ce66f-69e9-4a21-8e4f-ae677c46fdaa None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:52.769452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:52.796337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fb2d9-ca71-42c1-a371-212e630f4eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:52.797061 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fb2d9-ca71-42c1-a371-212e630f4eee None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:52.797359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fb2d9-ca71-42c1-a371-212e630f4eee None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:52.797612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fb2d9-ca71-42c1-a371-212e630f4eee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:52.873367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ce66f-69e9-4a21-8e4f-ae677c46fdaa None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:52.875549 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fb2d9-ca71-42c1-a371-212e630f4eee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:52.877074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fb2d9-ca71-42c1-a371-212e630f4eee None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:52.909387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11334b15-bdca-4732-8485-24f7dc34ac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:52.910279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11334b15-bdca-4732-8485-24f7dc34ac4e None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:52.910630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11334b15-bdca-4732-8485-24f7dc34ac4e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:52.910980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11334b15-bdca-4732-8485-24f7dc34ac4e None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:52.911768 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11334b15-bdca-4732-8485-24f7dc34ac4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:52.913262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11334b15-bdca-4732-8485-24f7dc34ac4e None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:52.946416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 513/1032] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:27:52 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:27:52.982659 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 520/1033] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:52 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:27:52.988338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11334b15-bdca-4732-8485-24f7dc34ac4e None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:53.043129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:53.104520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11334b15-bdca-4732-8485-24f7dc34ac4e None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:53.139825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742572f4-2903-4c05-bd77-7e4a65aebaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:53.140924 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742572f4-2903-4c05-bd77-7e4a65aebaaa None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:53.141207 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742572f4-2903-4c05-bd77-7e4a65aebaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:53.141480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742572f4-2903-4c05-bd77-7e4a65aebaaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:53.152827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742572f4-2903-4c05-bd77-7e4a65aebaaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:53.154326 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742572f4-2903-4c05-bd77-7e4a65aebaaa None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:53.164093 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454d5d-8db6-4b1c-a49d-6e5bc45883bd None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:53.164573 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454d5d-8db6-4b1c-a49d-6e5bc45883bd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:53.164649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454d5d-8db6-4b1c-a49d-6e5bc45883bd None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:53.168510 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 521/1034] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-742572f4-2903-4c05-bd77-7e4a65aebaaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:53.173478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:53.174390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:53.175308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:53.175308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:53.175308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:53.175308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:53.175536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf75e4-ff15-403c-9106-2a916ba1b312 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:53.175608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 514/1035] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:53.175685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf75e4-ff15-403c-9106-2a916ba1b312 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:53.175685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf75e4-ff15-403c-9106-2a916ba1b312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:53.180136 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742572f4-2903-4c05-bd77-7e4a65aebaaa None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:53.194413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b4bf75e4-ff15-403c-9106-2a916ba1b312 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:53.195606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b4bf75e4-ff15-403c-9106-2a916ba1b312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:53.197953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:53.274066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 522/1036] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:53.283462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:53.287808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742572f4-2903-4c05-bd77-7e4a65aebaaa None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:53.292823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf75e4-ff15-403c-9106-2a916ba1b312 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:53.326914 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104bf7a-7193-49a0-ba31-66521c650e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:53.327584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104bf7a-7193-49a0-ba31-66521c650e56 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:53.327869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104bf7a-7193-49a0-ba31-66521c650e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:53.328167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104bf7a-7193-49a0-ba31-66521c650e56 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:53.328892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104bf7a-7193-49a0-ba31-66521c650e56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:53.329199 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104bf7a-7193-49a0-ba31-66521c650e56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:53.330617 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104bf7a-7193-49a0-ba31-66521c650e56 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:53.347647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 515/1037] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:53.348704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104bf7a-7193-49a0-ba31-66521c650e56 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:53.401269 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf75e4-ff15-403c-9106-2a916ba1b312 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:53.438381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40793a-8cbe-4ed8-b709-17d3cedfeca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:53.439176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40793a-8cbe-4ed8-b709-17d3cedfeca3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:53.439461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40793a-8cbe-4ed8-b709-17d3cedfeca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:53.439737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40793a-8cbe-4ed8-b709-17d3cedfeca3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:53.440512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40793a-8cbe-4ed8-b709-17d3cedfeca3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:53.442144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40793a-8cbe-4ed8-b709-17d3cedfeca3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:53.449011 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104bf7a-7193-49a0-ba31-66521c650e56 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-1a40793a-8cbe-4ed8-b709-17d3cedfeca3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:53.454471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:53.455442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:27:53.456268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:53.460660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 523/1038] 10.222.0.60 () {56 vars in 1105 bytes} [Mon Jul 25 12:27:53 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:53.461882 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40793a-8cbe-4ed8-b709-17d3cedfeca3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:53.464706 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:53.487780 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0b6c-ca8c-4792-974e-dc8f6df88447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:53.488570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0b6c-ca8c-4792-974e-dc8f6df88447 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:53.488962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0b6c-ca8c-4792-974e-dc8f6df88447 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:53.489234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0b6c-ca8c-4792-974e-dc8f6df88447 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:53.496665 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a0b6c-ca8c-4792-974e-dc8f6df88447 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:53.498143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a0b6c-ca8c-4792-974e-dc8f6df88447 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-7e9a0b6c-ca8c-4792-974e-dc8f6df88447 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:53.511588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:53.512516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:53.513638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:53.513638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:53.513638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:53.513638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:53.513638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:53.513638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:53.513638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:53.513638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:53.513638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 516/1039] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:53.518357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0b6c-ca8c-4792-974e-dc8f6df88447 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:53.538272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:53.575106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40793a-8cbe-4ed8-b709-17d3cedfeca3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:53.610151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81934b4-6ac9-46b4-b99a-f718b7db4278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:53.610818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81934b4-6ac9-46b4-b99a-f718b7db4278 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:53.611105 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81934b4-6ac9-46b4-b99a-f718b7db4278 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:53.611383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81934b4-6ac9-46b4-b99a-f718b7db4278 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:53.612090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81934b4-6ac9-46b4-b99a-f718b7db4278 None admin] RBAC: Adding query filter params (name=image) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:53.612398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81934b4-6ac9-46b4-b99a-f718b7db4278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:53.613798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81934b4-6ac9-46b4-b99a-f718b7db4278 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:53.619635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 524/1040] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:53.625004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0b6c-ca8c-4792-974e-dc8f6df88447 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:53.625456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81934b4-6ac9-46b4-b99a-f718b7db4278 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:53.660256 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336d004-e4bc-4a44-b4a3-9c59b9a1b8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:53.660863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336d004-e4bc-4a44-b4a3-9c59b9a1b8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:53.661120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336d004-e4bc-4a44-b4a3-9c59b9a1b8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:53.661396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336d004-e4bc-4a44-b4a3-9c59b9a1b8e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:53.662265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336d004-e4bc-4a44-b4a3-9c59b9a1b8e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:53.662567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336d004-e4bc-4a44-b4a3-9c59b9a1b8e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:53.663985 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336d004-e4bc-4a44-b4a3-9c59b9a1b8e4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:53.677246 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 517/1041] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:53.683177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336d004-e4bc-4a44-b4a3-9c59b9a1b8e4 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:53.694237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:53.732505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81934b4-6ac9-46b4-b99a-f718b7db4278 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:53.765146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:53.769360 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac73cd-a7ea-493f-a1be-a69237df1e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:53.770048 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac73cd-a7ea-493f-a1be-a69237df1e2e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:53.770339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac73cd-a7ea-493f-a1be-a69237df1e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:53.770614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac73cd-a7ea-493f-a1be-a69237df1e2e None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:53.771324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac73cd-a7ea-493f-a1be-a69237df1e2e None admin] RBAC: Adding query filter params (type=image) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:53.771637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac73cd-a7ea-493f-a1be-a69237df1e2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:53.773210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac73cd-a7ea-493f-a1be-a69237df1e2e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:53.778414 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 525/1042] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:53.784106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac73cd-a7ea-493f-a1be-a69237df1e2e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:53.795134 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336d004-e4bc-4a44-b4a3-9c59b9a1b8e4 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:53.829593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc973c9-d817-4254-9473-ecf47e2e41a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:53.830576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc973c9-d817-4254-9473-ecf47e2e41a5 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:53.831091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc973c9-d817-4254-9473-ecf47e2e41a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:53.831370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc973c9-d817-4254-9473-ecf47e2e41a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:53.832332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc973c9-d817-4254-9473-ecf47e2e41a5 None admin] RBAC: Adding query filter params (role.id=a1464dd9ad2e4913b9d8cc6ba3c34207, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=ccf2dedaf7a449d2b51b9e9f3b420091) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:53.832516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc973c9-d817-4254-9473-ecf47e2e41a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:53.833903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc973c9-d817-4254-9473-ecf47e2e41a5 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:53.849943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 518/1043] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:53 2022] GET /identity/v3/role_assignments?user.id=ccf2dedaf7a449d2b51b9e9f3b420091&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=a1464dd9ad2e4913b9d8cc6ba3c34207 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:53.903484 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac73cd-a7ea-493f-a1be-a69237df1e2e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:53.934591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:53.941981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed02e2-cfac-4890-ba69-b0774e2a89be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:53.942933 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed02e2-cfac-4890-ba69-b0774e2a89be None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:53.943213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed02e2-cfac-4890-ba69-b0774e2a89be None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:53.943488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed02e2-cfac-4890-ba69-b0774e2a89be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:53.944286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed02e2-cfac-4890-ba69-b0774e2a89be None admin] RBAC: Adding query filter params (interface=public, service_id=e3aff7943b3949b388a477d772b246c1, region_id=RegionOne) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:53.944599 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed02e2-cfac-4890-ba69-b0774e2a89be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:53.946018 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed02e2-cfac-4890-ba69-b0774e2a89be None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:53.957223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 526/1044] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Jul 25 12:27:53 2022] GET /identity/v3/endpoints?service_id=e3aff7943b3949b388a477d772b246c1&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:53.958282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc973c9-d817-4254-9473-ecf47e2e41a5 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:54.065412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc973c9-d817-4254-9473-ecf47e2e41a5 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:54.099129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc86b4-1a96-41a4-86ac-5365fef4811f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:54.099824 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc86b4-1a96-41a4-86ac-5365fef4811f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:54.100108 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc86b4-1a96-41a4-86ac-5365fef4811f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:54.100385 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc86b4-1a96-41a4-86ac-5365fef4811f None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:54.101091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc86b4-1a96-41a4-86ac-5365fef4811f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:54.102543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc86b4-1a96-41a4-86ac-5365fef4811f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:54.110861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 519/1045] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Jul 25 12:27:53 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:27:55.006821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c3484-862a-4f2d-a1bd-f4d4e98bc93f None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:55.007335 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c3484-862a-4f2d-a1bd-f4d4e98bc93f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:55.007413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c3484-862a-4f2d-a1bd-f4d4e98bc93f None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:55.008886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 527/1046] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:55.014389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0746a34-86f3-4122-bd14-75cc4f2b4c6d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:55.014761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0746a34-86f3-4122-bd14-75cc4f2b4c6d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:55.014947 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0746a34-86f3-4122-bd14-75cc4f2b4c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:55.027154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:55.033061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-d0746a34-86f3-4122-bd14-75cc4f2b4c6d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:55.034267 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d0746a34-86f3-4122-bd14-75cc4f2b4c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:55.102114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 520/1047] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:55.125166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c3484-862a-4f2d-a1bd-f4d4e98bc93f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:55.235018 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c3484-862a-4f2d-a1bd-f4d4e98bc93f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:55.269947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f577b-7537-4208-8aa1-220fc726f484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:55.270728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f577b-7537-4208-8aa1-220fc726f484 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:55.271009 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f577b-7537-4208-8aa1-220fc726f484 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:55.271331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f577b-7537-4208-8aa1-220fc726f484 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:55.281336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f577b-7537-4208-8aa1-220fc726f484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-8d5f577b-7537-4208-8aa1-220fc726f484 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:55.288947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:55.290225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:55.291140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 528/1048] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:55.296123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0746a34-86f3-4122-bd14-75cc4f2b4c6d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:55.394956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0746a34-86f3-4122-bd14-75cc4f2b4c6d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:55.431889 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00915087-110f-492b-a575-20450002cae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:55.432560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00915087-110f-492b-a575-20450002cae0 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:55.432842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00915087-110f-492b-a575-20450002cae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:55.433119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00915087-110f-492b-a575-20450002cae0 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:55.433870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00915087-110f-492b-a575-20450002cae0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:55.434205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00915087-110f-492b-a575-20450002cae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:55.435560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00915087-110f-492b-a575-20450002cae0 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:55.437744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:55.448713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 521/1049] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:55.450577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f577b-7537-4208-8aa1-220fc726f484 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:55.506160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:55.551030 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f577b-7537-4208-8aa1-220fc726f484 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:55.588822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de7012-fbf5-438c-b7b6-3120b486ecde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:55.589703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de7012-fbf5-438c-b7b6-3120b486ecde None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:55.590012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de7012-fbf5-438c-b7b6-3120b486ecde None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:55.590290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de7012-fbf5-438c-b7b6-3120b486ecde None admin] PATH_INFO: `/v3/users/e457edb8f0544cc782fb98f6cd407d5a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:55.604542 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de7012-fbf5-438c-b7b6-3120b486ecde None admin] RBAC: Authorizing `identity:get_user(user_id=e457edb8f0544cc782fb98f6cd407d5a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:55.606018 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de7012-fbf5-438c-b7b6-3120b486ecde None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:55.608146 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 529/1050] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:27:55 2022] GET /identity/v3/users/e457edb8f0544cc782fb98f6cd407d5a => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:55.613835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00915087-110f-492b-a575-20450002cae0 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:55.715614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00915087-110f-492b-a575-20450002cae0 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:55.749812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e65e2-d44e-45ca-b17c-10dfaf8c8888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:55.750491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e65e2-d44e-45ca-b17c-10dfaf8c8888 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:55.750773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e65e2-d44e-45ca-b17c-10dfaf8c8888 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:55.751048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e65e2-d44e-45ca-b17c-10dfaf8c8888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:55.815123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:55.819394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e65e2-d44e-45ca-b17c-10dfaf8c8888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:55.821559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e65e2-d44e-45ca-b17c-10dfaf8c8888 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:55.827056 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f18981-ddd5-4852-93d5-9ddd3d15d477 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:55.827352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f18981-ddd5-4852-93d5-9ddd3d15d477 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:55.827631 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f18981-ddd5-4852-93d5-9ddd3d15d477 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:55.829139 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 530/1051] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:55.833980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9b754-1b66-437d-a3c2-e32d0e694cb4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:55.834271 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9b754-1b66-437d-a3c2-e32d0e694cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:55.834550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9b754-1b66-437d-a3c2-e32d0e694cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:55.858257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-20d9b754-1b66-437d-a3c2-e32d0e694cb4 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:55.859449 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-20d9b754-1b66-437d-a3c2-e32d0e694cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:55.926350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 522/1052] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:55 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:27:55.928336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 531/1053] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:55.932329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e65e2-d44e-45ca-b17c-10dfaf8c8888 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:55.950040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9b754-1b66-437d-a3c2-e32d0e694cb4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:55.996460 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:56.034058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:56.038142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e65e2-d44e-45ca-b17c-10dfaf8c8888 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:56.057704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9b754-1b66-437d-a3c2-e32d0e694cb4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:56.074719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54173b92-2098-42a5-9e08-5fc9825809bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:56.075507 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54173b92-2098-42a5-9e08-5fc9825809bb None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:56.075791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54173b92-2098-42a5-9e08-5fc9825809bb None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:56.076067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54173b92-2098-42a5-9e08-5fc9825809bb None admin] PATH_INFO: `/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:56.077240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54173b92-2098-42a5-9e08-5fc9825809bb None admin] RBAC: Authorizing `identity:get_project(project_id=cb9f9d8a6caf48c78015a46d8b038f75)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:56.078701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54173b92-2098-42a5-9e08-5fc9825809bb None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:56.080565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 523/1054] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:27:55 2022] GET /identity/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:56.086257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54173b92-2098-42a5-9e08-5fc9825809bb None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:56.095046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c536517-a3f4-4219-961e-8981219e0388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:56.096097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c536517-a3f4-4219-961e-8981219e0388 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:56.096586 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c536517-a3f4-4219-961e-8981219e0388 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:56.097052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c536517-a3f4-4219-961e-8981219e0388 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:56.108781 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c536517-a3f4-4219-961e-8981219e0388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-2c536517-a3f4-4219-961e-8981219e0388 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:56.117957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:56.119585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:56.120617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:56.120617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:56.120617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:56.120617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:56.120617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:56.120617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:56.120617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:56.120617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:56.120617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:56.120617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:56.120617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:27:56.120617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:56.123160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 532/1055] 10.222.0.60 () {56 vars in 1120 bytes} [Mon Jul 25 12:27:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:56.129198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c536517-a3f4-4219-961e-8981219e0388 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:56.190911 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54173b92-2098-42a5-9e08-5fc9825809bb None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:56.224068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240861f6-a67c-4267-88d7-a716f66e9cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:56.224717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240861f6-a67c-4267-88d7-a716f66e9cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:56.224988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240861f6-a67c-4267-88d7-a716f66e9cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:56.225266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240861f6-a67c-4267-88d7-a716f66e9cb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:56.226131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240861f6-a67c-4267-88d7-a716f66e9cb7 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=cb9f9d8a6caf48c78015a46d8b038f75, user.id=e457edb8f0544cc782fb98f6cd407d5a) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:56.226440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240861f6-a67c-4267-88d7-a716f66e9cb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:56.228382 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240861f6-a67c-4267-88d7-a716f66e9cb7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:56.231599 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c536517-a3f4-4219-961e-8981219e0388 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:56.233097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:56.234015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 524/1056] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:27:56 2022] GET /identity/v3/role_assignments?user.id=e457edb8f0544cc782fb98f6cd407d5a&scope.project.id=cb9f9d8a6caf48c78015a46d8b038f75&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:56.239453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d32f8a-47a2-419b-ba42-1b61806629ce None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:56.239761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d32f8a-47a2-419b-ba42-1b61806629ce None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:56.240017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d32f8a-47a2-419b-ba42-1b61806629ce None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:56.241594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 525/1057] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:56.248022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e66b59-bc52-42ed-a242-47fac942976d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:56.248334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e66b59-bc52-42ed-a242-47fac942976d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:56.248590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e66b59-bc52-42ed-a242-47fac942976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:56.267807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-49e66b59-bc52-42ed-a242-47fac942976d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:56.269041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-49e66b59-bc52-42ed-a242-47fac942976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:56.270903 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e74a7-dfc8-4ac4-ad88-a6afc65bd9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:56.271598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e74a7-dfc8-4ac4-ad88-a6afc65bd9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:56.271891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e74a7-dfc8-4ac4-ad88-a6afc65bd9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:56.272172 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e74a7-dfc8-4ac4-ad88-a6afc65bd9b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:56.272976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e74a7-dfc8-4ac4-ad88-a6afc65bd9b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:56.273285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e74a7-dfc8-4ac4-ad88-a6afc65bd9b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:56.274754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e74a7-dfc8-4ac4-ad88-a6afc65bd9b6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:56.284458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 533/1058] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Jul 25 12:27:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:56.290999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e74a7-dfc8-4ac4-ad88-a6afc65bd9b6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:56.297963 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:56.346595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 526/1059] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:56.373379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e66b59-bc52-42ed-a242-47fac942976d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:56.405499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e74a7-dfc8-4ac4-ad88-a6afc65bd9b6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:56.443621 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1c821-db15-485c-a38f-792abcc55fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:56.445040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1c821-db15-485c-a38f-792abcc55fbf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:56.445433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1c821-db15-485c-a38f-792abcc55fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:56.445776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1c821-db15-485c-a38f-792abcc55fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:56.457653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1c821-db15-485c-a38f-792abcc55fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:56.459141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1c821-db15-485c-a38f-792abcc55fbf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c9f1c821-db15-485c-a38f-792abcc55fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:56.476842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:56.478393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:56.479837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:56.479837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:56.479837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:56.479837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:56.479837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 534/1060] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:56.484425 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1c821-db15-485c-a38f-792abcc55fbf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:56.488548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e66b59-bc52-42ed-a242-47fac942976d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:56.503250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:56.526764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43472ef6-5e5a-4472-bb01-717869457569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:56.527565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43472ef6-5e5a-4472-bb01-717869457569 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:56.527848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43472ef6-5e5a-4472-bb01-717869457569 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:56.528123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43472ef6-5e5a-4472-bb01-717869457569 None admin] PATH_INFO: `/v3/services/image` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:56.528782 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43472ef6-5e5a-4472-bb01-717869457569 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:56.530171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43472ef6-5e5a-4472-bb01-717869457569 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-43472ef6-5e5a-4472-bb01-717869457569 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:56.542289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:56.543264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:56.544242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 527/1061] 10.222.0.60 () {56 vars in 1105 bytes} [Mon Jul 25 12:27:56 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:27:56.549344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43472ef6-5e5a-4472-bb01-717869457569 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:56.576156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:56.596383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1c821-db15-485c-a38f-792abcc55fbf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:56.633355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237078cc-0859-4f48-bdcc-2a86f0d3f3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:56.634644 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237078cc-0859-4f48-bdcc-2a86f0d3f3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:56.634927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237078cc-0859-4f48-bdcc-2a86f0d3f3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:56.635311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237078cc-0859-4f48-bdcc-2a86f0d3f3d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:56.636025 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237078cc-0859-4f48-bdcc-2a86f0d3f3d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:56.636359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237078cc-0859-4f48-bdcc-2a86f0d3f3d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:56.637798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237078cc-0859-4f48-bdcc-2a86f0d3f3d4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:56.653115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43472ef6-5e5a-4472-bb01-717869457569 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:56.655470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 535/1062] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:56.656341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237078cc-0859-4f48-bdcc-2a86f0d3f3d4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:56.689293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2213c2c-e16f-4b10-a218-35f84f9b2c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:56.690473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2213c2c-e16f-4b10-a218-35f84f9b2c48 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:56.690969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2213c2c-e16f-4b10-a218-35f84f9b2c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:56.691283 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2213c2c-e16f-4b10-a218-35f84f9b2c48 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:56.691971 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2213c2c-e16f-4b10-a218-35f84f9b2c48 None admin] RBAC: Adding query filter params (name=image) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:56.692277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2213c2c-e16f-4b10-a218-35f84f9b2c48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:56.693645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2213c2c-e16f-4b10-a218-35f84f9b2c48 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:56.701088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 528/1063] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:56.706364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2213c2c-e16f-4b10-a218-35f84f9b2c48 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:56.753015 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:56.776820 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237078cc-0859-4f48-bdcc-2a86f0d3f3d4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:56.798844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:56.814269 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759f204-b67d-421b-a4ce-fe9e8b2f2aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:56.815127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759f204-b67d-421b-a4ce-fe9e8b2f2aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:56.815252 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2213c2c-e16f-4b10-a218-35f84f9b2c48 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:56.815598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759f204-b67d-421b-a4ce-fe9e8b2f2aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:56.821391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759f204-b67d-421b-a4ce-fe9e8b2f2aa6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:56.831005 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759f204-b67d-421b-a4ce-fe9e8b2f2aa6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:56.832578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759f204-b67d-421b-a4ce-fe9e8b2f2aa6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c759f204-b67d-421b-a4ce-fe9e8b2f2aa6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:27:56.848637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:56.849600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:56.850603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:56.850603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:56.850603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:27:56.850603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:27:56.850603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:27:56.850603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:27:56.850603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:27:56.850603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:27:56.850603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:27:56.850603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:56.851138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637816f-2459-4505-b518-c8f1a949464c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:56.851138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637816f-2459-4505-b518-c8f1a949464c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:56.851138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637816f-2459-4505-b518-c8f1a949464c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:56.851138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637816f-2459-4505-b518-c8f1a949464c None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:56.851412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637816f-2459-4505-b518-c8f1a949464c None admin] RBAC: Adding query filter params (type=image) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:56.851412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637816f-2459-4505-b518-c8f1a949464c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:56.852845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637816f-2459-4505-b518-c8f1a949464c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:56.854862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 536/1064] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Jul 25 12:27:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:56.855686 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759f204-b67d-421b-a4ce-fe9e8b2f2aa6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:56.862975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 529/1065] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:27:56.863707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637816f-2459-4505-b518-c8f1a949464c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:56.964731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759f204-b67d-421b-a4ce-fe9e8b2f2aa6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:56.965576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637816f-2459-4505-b518-c8f1a949464c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:57.000556 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff55238-3ded-4952-a098-4b6512970bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:57.000766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d437a8-aeee-484c-8f04-ab5ed260a035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:57.001208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff55238-3ded-4952-a098-4b6512970bd7 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:57.001458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d437a8-aeee-484c-8f04-ab5ed260a035 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:57.001566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff55238-3ded-4952-a098-4b6512970bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:57.001750 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff55238-3ded-4952-a098-4b6512970bd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:57.001889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d437a8-aeee-484c-8f04-ab5ed260a035 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:57.002250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d437a8-aeee-484c-8f04-ab5ed260a035 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:57.002516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff55238-3ded-4952-a098-4b6512970bd7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:57.003529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d437a8-aeee-484c-8f04-ab5ed260a035 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:57.003897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d437a8-aeee-484c-8f04-ab5ed260a035 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:57.004394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff55238-3ded-4952-a098-4b6512970bd7 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:57.005419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d437a8-aeee-484c-8f04-ab5ed260a035 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:57.008363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:57.028427 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 537/1066] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Jul 25 12:27:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:57.035712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d437a8-aeee-484c-8f04-ab5ed260a035 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:57.038209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 530/1067] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:27:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:27:57.147105 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d437a8-aeee-484c-8f04-ab5ed260a035 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:57.194175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223c5d7-46a8-45b4-8b71-b9257476fd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:57.194416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223c5d7-46a8-45b4-8b71-b9257476fd8c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:57.194687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223c5d7-46a8-45b4-8b71-b9257476fd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:57.194962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223c5d7-46a8-45b4-8b71-b9257476fd8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:57.206003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223c5d7-46a8-45b4-8b71-b9257476fd8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:57.207390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223c5d7-46a8-45b4-8b71-b9257476fd8c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a223c5d7-46a8-45b4-8b71-b9257476fd8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:57.225069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:57.226024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:57.226929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:57.226929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:57.226929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:57.226929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:57.226929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 538/1068] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:57.232842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff55238-3ded-4952-a098-4b6512970bd7 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:57.245884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:57.343311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff55238-3ded-4952-a098-4b6512970bd7 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:57.377578 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086e82d-339a-43f2-afa7-769a2a74597c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:57.378303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086e82d-339a-43f2-afa7-769a2a74597c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:57.378591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086e82d-339a-43f2-afa7-769a2a74597c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:57.378866 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086e82d-339a-43f2-afa7-769a2a74597c None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:57.379587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086e82d-339a-43f2-afa7-769a2a74597c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:57.379895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086e82d-339a-43f2-afa7-769a2a74597c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:57.381254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086e82d-339a-43f2-afa7-769a2a74597c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:57.397651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 531/1069] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:57.399072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223c5d7-46a8-45b4-8b71-b9257476fd8c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:57.443225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:57.502560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223c5d7-46a8-45b4-8b71-b9257476fd8c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:57.537090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6eb94e-c0b8-4a18-a9cb-01e016625887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:57.537880 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6eb94e-c0b8-4a18-a9cb-01e016625887 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:57.538182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6eb94e-c0b8-4a18-a9cb-01e016625887 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:57.538459 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6eb94e-c0b8-4a18-a9cb-01e016625887 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:57.545654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6eb94e-c0b8-4a18-a9cb-01e016625887 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:57.547123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6eb94e-c0b8-4a18-a9cb-01e016625887 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-7e6eb94e-c0b8-4a18-a9cb-01e016625887 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:27:57.560807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:57.561762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:57.562720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:27:57.562720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:27:57.562720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:27:57.562720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:27:57.562720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:27:57.562720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:27:57.562720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:27:57.562720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:57.566229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 539/1070] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:27:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:57.567794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086e82d-339a-43f2-afa7-769a2a74597c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:57.635949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:57.670446 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086e82d-339a-43f2-afa7-769a2a74597c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:57.704333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e714f2-61c3-49ff-aa05-69a1b37ac5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:57.704990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e714f2-61c3-49ff-aa05-69a1b37ac5ad None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:57.705272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e714f2-61c3-49ff-aa05-69a1b37ac5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:57.705545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e714f2-61c3-49ff-aa05-69a1b37ac5ad None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:57.706351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e714f2-61c3-49ff-aa05-69a1b37ac5ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:57.706659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e714f2-61c3-49ff-aa05-69a1b37ac5ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:57.708083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e714f2-61c3-49ff-aa05-69a1b37ac5ad None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:57.725122 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 532/1071] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:27:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:57.726864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6eb94e-c0b8-4a18-a9cb-01e016625887 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:57.829521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6eb94e-c0b8-4a18-a9cb-01e016625887 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:57.831068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:57.864727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabcd77-f90b-4941-a646-e62bf6270fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:57.865671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabcd77-f90b-4941-a646-e62bf6270fc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:57.865982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabcd77-f90b-4941-a646-e62bf6270fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:57.866260 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabcd77-f90b-4941-a646-e62bf6270fc4 None admin] PATH_INFO: `/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/ccf2dedaf7a449d2b51b9e9f3b420091/roles/a1464dd9ad2e4913b9d8cc6ba3c34207` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:57.868502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabcd77-f90b-4941-a646-e62bf6270fc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff5a41f8ca6438295c512792d2022e4, user_id=ccf2dedaf7a449d2b51b9e9f3b420091, role_id=a1464dd9ad2e4913b9d8cc6ba3c34207)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:57.870014 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabcd77-f90b-4941-a646-e62bf6270fc4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:57.884906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 540/1072] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:27:57 2022] PUT /identity/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/ccf2dedaf7a449d2b51b9e9f3b420091/roles/a1464dd9ad2e4913b9d8cc6ba3c34207 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:27:58.232828 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fbd26-831a-4e6e-8946-fb96c5c57927 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:58.233362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fbd26-831a-4e6e-8946-fb96c5c57927 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:58.233362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fbd26-831a-4e6e-8946-fb96c5c57927 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:58.234839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 533/1073] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:58.240039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b261fe1-81fc-4079-b21e-d092a923d95f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:58.240439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b261fe1-81fc-4079-b21e-d092a923d95f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:58.240605 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b261fe1-81fc-4079-b21e-d092a923d95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:58.260100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9b261fe1-81fc-4079-b21e-d092a923d95f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:58.261266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9b261fe1-81fc-4079-b21e-d092a923d95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:58.329314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 541/1074] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:27:58.350825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fbd26-831a-4e6e-8946-fb96c5c57927 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:58.454207 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fbd26-831a-4e6e-8946-fb96c5c57927 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:58.479347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:58.488550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d96939-420e-4c9a-871f-76a3678221b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:58.489338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d96939-420e-4c9a-871f-76a3678221b9 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:58.489614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d96939-420e-4c9a-871f-76a3678221b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:58.489882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d96939-420e-4c9a-871f-76a3678221b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:58.501754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d96939-420e-4c9a-871f-76a3678221b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-17d96939-420e-4c9a-871f-76a3678221b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:27:58.510046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:58.510967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:58.511814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:58.511814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:58.511814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:58.511814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:58.511814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:27:58.511814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:27:58.511814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:27:58.511814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:27:58.511814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:27:58.511814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:27:58.511814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:27:58.511814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:27:58.514303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 534/1075] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:27:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:58.517215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b261fe1-81fc-4079-b21e-d092a923d95f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:58.607773 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:58.618251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b261fe1-81fc-4079-b21e-d092a923d95f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:58.652492 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6947ac-7c96-496b-b979-0f139c03cbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:58.653161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6947ac-7c96-496b-b979-0f139c03cbdf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:58.653441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6947ac-7c96-496b-b979-0f139c03cbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:58.653714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6947ac-7c96-496b-b979-0f139c03cbdf None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:58.654481 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6947ac-7c96-496b-b979-0f139c03cbdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:58.654786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6947ac-7c96-496b-b979-0f139c03cbdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:58.656145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6947ac-7c96-496b-b979-0f139c03cbdf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:58.664429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 542/1076] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:27:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:58.670601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d96939-420e-4c9a-871f-76a3678221b9 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:58.771458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d96939-420e-4c9a-871f-76a3678221b9 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:58.806019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0a1bd-5880-49ea-9b01-5531bd32ddf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:58.806883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0a1bd-5880-49ea-9b01-5531bd32ddf6 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:58.807162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0a1bd-5880-49ea-9b01-5531bd32ddf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:58.807434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0a1bd-5880-49ea-9b01-5531bd32ddf6 None admin] PATH_INFO: `/v3/users/e457edb8f0544cc782fb98f6cd407d5a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:58.808619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0a1bd-5880-49ea-9b01-5531bd32ddf6 None admin] RBAC: Authorizing `identity:get_user(user_id=e457edb8f0544cc782fb98f6cd407d5a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:58.810071 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0a1bd-5880-49ea-9b01-5531bd32ddf6 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:58.816351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 535/1077] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:27:58 2022] GET /identity/v3/users/e457edb8f0544cc782fb98f6cd407d5a => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:58.817500 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6947ac-7c96-496b-b979-0f139c03cbdf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:58.921213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6947ac-7c96-496b-b979-0f139c03cbdf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:58.958722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cd407-e1ec-4fab-b945-5322bed05c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:58.959585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cd407-e1ec-4fab-b945-5322bed05c29 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:58.959869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cd407-e1ec-4fab-b945-5322bed05c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:58.960143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cd407-e1ec-4fab-b945-5322bed05c29 None admin] PATH_INFO: `/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:58.961339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cd407-e1ec-4fab-b945-5322bed05c29 None admin] RBAC: Authorizing `identity:get_project(project_id=cb9f9d8a6caf48c78015a46d8b038f75)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:58.963358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cd407-e1ec-4fab-b945-5322bed05c29 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:58.965228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 543/1078] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:27:58 2022] GET /identity/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:58.971317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0a1bd-5880-49ea-9b01-5531bd32ddf6 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:59.032276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:59.078231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0a1bd-5880-49ea-9b01-5531bd32ddf6 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:59.104130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b39a3-09aa-4d7b-9bbb-0d193cc5db96 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:59.104428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b39a3-09aa-4d7b-9bbb-0d193cc5db96 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:59.104706 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b39a3-09aa-4d7b-9bbb-0d193cc5db96 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:59.106092 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 544/1079] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:27:59.111053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28143116-d8c3-4c95-900a-4086bc7a02fe None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:59.111341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28143116-d8c3-4c95-900a-4086bc7a02fe None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:59.111624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28143116-d8c3-4c95-900a-4086bc7a02fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:59.114083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e435f28-1df3-444d-b0e1-8a52755dc072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:59.115171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e435f28-1df3-444d-b0e1-8a52755dc072 None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:59.115623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e435f28-1df3-444d-b0e1-8a52755dc072 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:59.116072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e435f28-1df3-444d-b0e1-8a52755dc072 None admin] PATH_INFO: `/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75/users/e457edb8f0544cc782fb98f6cd407d5a/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:59.118482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e435f28-1df3-444d-b0e1-8a52755dc072 None admin] RBAC: Authorizing `identity:create_grant(project_id=cb9f9d8a6caf48c78015a46d8b038f75, user_id=e457edb8f0544cc782fb98f6cd407d5a, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:59.120958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e435f28-1df3-444d-b0e1-8a52755dc072 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:59.130384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-28143116-d8c3-4c95-900a-4086bc7a02fe None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:59.131446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-28143116-d8c3-4c95-900a-4086bc7a02fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:59.136695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 536/1080] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:27:58 2022] PUT /identity/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75/users/e457edb8f0544cc782fb98f6cd407d5a/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:27:59.154025 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:59.221942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 545/1081] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:27:59.241796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e435f28-1df3-444d-b0e1-8a52755dc072 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:59.351162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e435f28-1df3-444d-b0e1-8a52755dc072 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:59.367755 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:59.388120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730dd1f-e65d-4fe8-8774-ed2ba7b4c6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:59.388774 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730dd1f-e65d-4fe8-8774-ed2ba7b4c6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:59.389056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730dd1f-e65d-4fe8-8774-ed2ba7b4c6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:59.389331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730dd1f-e65d-4fe8-8774-ed2ba7b4c6f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:59.455971 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730dd1f-e65d-4fe8-8774-ed2ba7b4c6f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:59.457422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730dd1f-e65d-4fe8-8774-ed2ba7b4c6f9 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:59.560808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 537/1082] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:27:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:27:59.562460 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28143116-d8c3-4c95-900a-4086bc7a02fe None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:59.668577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28143116-d8c3-4c95-900a-4086bc7a02fe None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:59.703116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d379db4-a2f9-482b-b48d-abf548090ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:59.703905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d379db4-a2f9-482b-b48d-abf548090ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:59.704180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d379db4-a2f9-482b-b48d-abf548090ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:59.704447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d379db4-a2f9-482b-b48d-abf548090ff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:59.714315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d379db4-a2f9-482b-b48d-abf548090ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:59.716028 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d379db4-a2f9-482b-b48d-abf548090ff9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-0d379db4-a2f9-482b-b48d-abf548090ff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:27:59.733297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:27:59.734465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:27:59.735451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:27:59.735451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:27:59.735451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:27:59.735451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:27:59.735451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 546/1083] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:27:59.741961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730dd1f-e65d-4fe8-8774-ed2ba7b4c6f9 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:59.759731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:59.844417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730dd1f-e65d-4fe8-8774-ed2ba7b4c6f9 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:59.878581 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a977c8-8060-458f-8961-1c710f22cc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:59.879240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a977c8-8060-458f-8961-1c710f22cc5d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:59.879521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a977c8-8060-458f-8961-1c710f22cc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:59.879801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a977c8-8060-458f-8961-1c710f22cc5d None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:59.880522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a977c8-8060-458f-8961-1c710f22cc5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:27:59.880830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a977c8-8060-458f-8961-1c710f22cc5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:59.882212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a977c8-8060-458f-8961-1c710f22cc5d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:59.890560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d458a13-22b0-41bd-acb3-1e5b5b6e83e6 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:59.890848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d458a13-22b0-41bd-acb3-1e5b5b6e83e6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:59.891125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d458a13-22b0-41bd-acb3-1e5b5b6e83e6 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:59.892712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 547/1084] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:27:59.895033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 538/1085] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:27:59.897268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b3d3b-28bd-4268-beb8-053b1b002291 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:59.897555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b3d3b-28bd-4268-beb8-053b1b002291 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:59.897833 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b3d3b-28bd-4268-beb8-053b1b002291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:59.906408 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:59.918396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f39b3d3b-28bd-4268-beb8-053b1b002291 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:59.919674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f39b3d3b-28bd-4268-beb8-053b1b002291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:59.994285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 548/1086] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:27:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:28:00.017292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a977c8-8060-458f-8961-1c710f22cc5d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:00.097327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:00.122609 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a977c8-8060-458f-8961-1c710f22cc5d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:00.157232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0056a6a-6d08-43bf-941f-489afd3a1582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:00.158112 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0056a6a-6d08-43bf-941f-489afd3a1582 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:00.158394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0056a6a-6d08-43bf-941f-489afd3a1582 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:00.158670 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0056a6a-6d08-43bf-941f-489afd3a1582 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:00.173835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0056a6a-6d08-43bf-941f-489afd3a1582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-d0056a6a-6d08-43bf-941f-489afd3a1582 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:28:00.182790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:00.183741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:28:00.184573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 539/1087] 10.222.0.60 () {56 vars in 1120 bytes} [Mon Jul 25 12:28:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:28:00.191336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b3d3b-28bd-4268-beb8-053b1b002291 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:00.294458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b3d3b-28bd-4268-beb8-053b1b002291 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:00.328470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822e9c7-0ac0-496c-918e-74880ee07326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:00.329135 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822e9c7-0ac0-496c-918e-74880ee07326 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:00.329420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822e9c7-0ac0-496c-918e-74880ee07326 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:00.329694 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822e9c7-0ac0-496c-918e-74880ee07326 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:00.330471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822e9c7-0ac0-496c-918e-74880ee07326 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:00.330777 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822e9c7-0ac0-496c-918e-74880ee07326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:00.332156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822e9c7-0ac0-496c-918e-74880ee07326 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:00.333473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:00.340992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 549/1088] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Jul 25 12:28:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:00.346634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0056a6a-6d08-43bf-941f-489afd3a1582 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:00.449228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0056a6a-6d08-43bf-941f-489afd3a1582 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:00.480237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982a607-676f-4943-b452-e690b2d60547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:00.481009 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a607-676f-4943-b452-e690b2d60547 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:00.481538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a607-676f-4943-b452-e690b2d60547 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:00.481809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a607-676f-4943-b452-e690b2d60547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:00.484400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:00.492722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982a607-676f-4943-b452-e690b2d60547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:00.494705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982a607-676f-4943-b452-e690b2d60547 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-1982a607-676f-4943-b452-e690b2d60547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:28:00.511419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:00.512464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:00.518188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:28:00.518188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:28:00.518188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:28:00.518188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:28:00.518188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 540/1089] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:28:00.519078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822e9c7-0ac0-496c-918e-74880ee07326 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:00.617091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822e9c7-0ac0-496c-918e-74880ee07326 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:00.650433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96df37-a57e-4fb9-9bf5-457076812c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:00.651088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96df37-a57e-4fb9-9bf5-457076812c44 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:00.651370 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96df37-a57e-4fb9-9bf5-457076812c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:00.651646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96df37-a57e-4fb9-9bf5-457076812c44 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:00.652387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96df37-a57e-4fb9-9bf5-457076812c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:00.652692 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96df37-a57e-4fb9-9bf5-457076812c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:00.654079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96df37-a57e-4fb9-9bf5-457076812c44 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:00.665983 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 550/1090] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:00.671522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982a607-676f-4943-b452-e690b2d60547 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:00.768547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982a607-676f-4943-b452-e690b2d60547 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:00.801120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8776e3-e623-4c60-af17-79a0f1a2d974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:00.801903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8776e3-e623-4c60-af17-79a0f1a2d974 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:00.802210 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8776e3-e623-4c60-af17-79a0f1a2d974 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:00.802487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8776e3-e623-4c60-af17-79a0f1a2d974 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:00.810381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8776e3-e623-4c60-af17-79a0f1a2d974 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:00.811798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8776e3-e623-4c60-af17-79a0f1a2d974 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-bf8776e3-e623-4c60-af17-79a0f1a2d974 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:28:00.825371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:00.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:00.827282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:00.827282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:00.827282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:28:00.827282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:28:00.827282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:28:00.827282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:28:00.827282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:28:00.827282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:28:00.827282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:28:00.827282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:28:00.827282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 541/1091] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Jul 25 12:28:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:28:00.832247 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96df37-a57e-4fb9-9bf5-457076812c44 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:00.888212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:00.931768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96df37-a57e-4fb9-9bf5-457076812c44 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:00.964331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50438f4-19e6-483e-925e-b0fe7ae772a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:00.964990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50438f4-19e6-483e-925e-b0fe7ae772a6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:00.965270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50438f4-19e6-483e-925e-b0fe7ae772a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:00.965542 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50438f4-19e6-483e-925e-b0fe7ae772a6 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:00.966396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50438f4-19e6-483e-925e-b0fe7ae772a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:00.966701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50438f4-19e6-483e-925e-b0fe7ae772a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:00.968113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50438f4-19e6-483e-925e-b0fe7ae772a6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:00.985379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 551/1092] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Jul 25 12:28:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:00.990801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8776e3-e623-4c60-af17-79a0f1a2d974 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:01.038389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:01.088496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8776e3-e623-4c60-af17-79a0f1a2d974 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:01.120892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55c4f0-12d6-4821-8569-3f0d1b26aad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:01.121532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55c4f0-12d6-4821-8569-3f0d1b26aad4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:01.121811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55c4f0-12d6-4821-8569-3f0d1b26aad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:01.122108 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55c4f0-12d6-4821-8569-3f0d1b26aad4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:01.171901 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881f770-72e8-4d42-988a-d2c018dbd402 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:01.172190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881f770-72e8-4d42-988a-d2c018dbd402 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:01.172470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881f770-72e8-4d42-988a-d2c018dbd402 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:01.173866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 552/1093] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:28:01.178936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5394d-1433-4ebe-b501-7d7c3e28a625 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:01.179227 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5394d-1433-4ebe-b501-7d7c3e28a625 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:01.179503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5394d-1433-4ebe-b501-7d7c3e28a625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:01.190520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55c4f0-12d6-4821-8569-3f0d1b26aad4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:01.191960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55c4f0-12d6-4821-8569-3f0d1b26aad4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:01.197413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d1d5394d-1433-4ebe-b501-7d7c3e28a625 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:01.198645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d1d5394d-1433-4ebe-b501-7d7c3e28a625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:01.238938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:01.265446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 553/1094] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:28:01.288657 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5394d-1433-4ebe-b501-7d7c3e28a625 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:01.289284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 542/1095] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:28:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:28:01.295242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55c4f0-12d6-4821-8569-3f0d1b26aad4 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:01.333314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:01.387827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5394d-1433-4ebe-b501-7d7c3e28a625 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:01.390537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55c4f0-12d6-4821-8569-3f0d1b26aad4 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:01.421933 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362f191-3ac0-4fe3-b5ae-bc8fb0ad6c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:01.422742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362f191-3ac0-4fe3-b5ae-bc8fb0ad6c5c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:01.423029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362f191-3ac0-4fe3-b5ae-bc8fb0ad6c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:01.423308 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362f191-3ac0-4fe3-b5ae-bc8fb0ad6c5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:01.423404 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32851cd-fb72-4941-82ba-067f72968d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:01.424089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32851cd-fb72-4941-82ba-067f72968d82 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:01.424368 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32851cd-fb72-4941-82ba-067f72968d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:01.424640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32851cd-fb72-4941-82ba-067f72968d82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:01.433260 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362f191-3ac0-4fe3-b5ae-bc8fb0ad6c5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:28:01.434496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32851cd-fb72-4941-82ba-067f72968d82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:01.435928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32851cd-fb72-4941-82ba-067f72968d82 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-a362f191-3ac0-4fe3-b5ae-bc8fb0ad6c5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:28:01.441006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:01.442003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:28:01.442853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 554/1096] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:28:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:28:01.445517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:01.448803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362f191-3ac0-4fe3-b5ae-bc8fb0ad6c5c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-c32851cd-fb72-4941-82ba-067f72968d82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:28:01.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:01.457362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:01.458350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:28:01.458350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:28:01.458350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:28:01.458350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:28:01.462657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 543/1097] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:28:01.463460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32851cd-fb72-4941-82ba-067f72968d82 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:01.546564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362f191-3ac0-4fe3-b5ae-bc8fb0ad6c5c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:01.548736 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:01.560411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32851cd-fb72-4941-82ba-067f72968d82 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:01.580145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06400007-0e2a-4f60-aa8d-1b908b854699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:01.580810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06400007-0e2a-4f60-aa8d-1b908b854699 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:01.581084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06400007-0e2a-4f60-aa8d-1b908b854699 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:01.581353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06400007-0e2a-4f60-aa8d-1b908b854699 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:01.582125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06400007-0e2a-4f60-aa8d-1b908b854699 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:01.582431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06400007-0e2a-4f60-aa8d-1b908b854699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:01.583785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06400007-0e2a-4f60-aa8d-1b908b854699 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:01.592072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 555/1098] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:28:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:28:01.595786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd701f8-1283-4fdd-b5cc-62ec5cffb6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:01.596434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd701f8-1283-4fdd-b5cc-62ec5cffb6ec None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:01.596960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd701f8-1283-4fdd-b5cc-62ec5cffb6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:01.596960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd701f8-1283-4fdd-b5cc-62ec5cffb6ec None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:01.597675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd701f8-1283-4fdd-b5cc-62ec5cffb6ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:01.597774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06400007-0e2a-4f60-aa8d-1b908b854699 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:01.598004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd701f8-1283-4fdd-b5cc-62ec5cffb6ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:01.599344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd701f8-1283-4fdd-b5cc-62ec5cffb6ec None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:01.611023 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 544/1099] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:01.616061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd701f8-1283-4fdd-b5cc-62ec5cffb6ec None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:01.672501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:01.704262 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06400007-0e2a-4f60-aa8d-1b908b854699 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:01.713631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd701f8-1283-4fdd-b5cc-62ec5cffb6ec None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:01.737645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddb05e-5f17-47fa-b2a5-510bd7f576af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:01.738559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddb05e-5f17-47fa-b2a5-510bd7f576af None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:01.738842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddb05e-5f17-47fa-b2a5-510bd7f576af None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:01.739117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddb05e-5f17-47fa-b2a5-510bd7f576af None admin] PATH_INFO: `/v3/users/e457edb8f0544cc782fb98f6cd407d5a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:01.740309 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddb05e-5f17-47fa-b2a5-510bd7f576af None admin] RBAC: Authorizing `identity:get_user(user_id=e457edb8f0544cc782fb98f6cd407d5a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:01.741740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddb05e-5f17-47fa-b2a5-510bd7f576af None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:01.744098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 556/1100] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:01 2022] GET /identity/v3/users/e457edb8f0544cc782fb98f6cd407d5a => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:28:01.747502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a55b3-0daf-4ac2-b95e-8c54ed261724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:01.748289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a55b3-0daf-4ac2-b95e-8c54ed261724 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:01.748759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a55b3-0daf-4ac2-b95e-8c54ed261724 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:01.748864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a55b3-0daf-4ac2-b95e-8c54ed261724 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:01.749560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddb05e-5f17-47fa-b2a5-510bd7f576af None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:01.755734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a55b3-0daf-4ac2-b95e-8c54ed261724 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:01.757154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a55b3-0daf-4ac2-b95e-8c54ed261724 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-925a55b3-0daf-4ac2-b95e-8c54ed261724 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:28:01.771636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:01.772708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:01.773902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:28:01.773902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:28:01.773902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:28:01.773902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:28:01.773902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:28:01.773902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:28:01.773902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:28:01.773902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:28:01.773902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 545/1101] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:28:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:28:01.778875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a55b3-0daf-4ac2-b95e-8c54ed261724 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:01.786341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:01.852116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddb05e-5f17-47fa-b2a5-510bd7f576af None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:01.875672 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a55b3-0daf-4ac2-b95e-8c54ed261724 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:01.885586 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8da07d-fbac-41a1-8dcb-e276ba4e0d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:01.886263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8da07d-fbac-41a1-8dcb-e276ba4e0d63 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:01.886545 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8da07d-fbac-41a1-8dcb-e276ba4e0d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:01.886826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8da07d-fbac-41a1-8dcb-e276ba4e0d63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:01.895714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:01.910546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c369cad-6472-4ba6-9d25-39c80c70900d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:01.911221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c369cad-6472-4ba6-9d25-39c80c70900d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:01.911501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c369cad-6472-4ba6-9d25-39c80c70900d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:01.911777 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c369cad-6472-4ba6-9d25-39c80c70900d None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:01.912552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c369cad-6472-4ba6-9d25-39c80c70900d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:01.912860 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c369cad-6472-4ba6-9d25-39c80c70900d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:01.914291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c369cad-6472-4ba6-9d25-39c80c70900d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:01.926594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 546/1102] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:28:01.932580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c369cad-6472-4ba6-9d25-39c80c70900d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:01.953666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8da07d-fbac-41a1-8dcb-e276ba4e0d63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:01.955142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8da07d-fbac-41a1-8dcb-e276ba4e0d63 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:01.987804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:02.029299 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c369cad-6472-4ba6-9d25-39c80c70900d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:02.052960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 557/1103] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:28:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:28:02.058783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8da07d-fbac-41a1-8dcb-e276ba4e0d63 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:02.062531 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491b5bd-e01b-4df1-82b8-b616f586f8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:02.063178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491b5bd-e01b-4df1-82b8-b616f586f8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:02.063458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491b5bd-e01b-4df1-82b8-b616f586f8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:02.063732 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491b5bd-e01b-4df1-82b8-b616f586f8e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:02.064550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491b5bd-e01b-4df1-82b8-b616f586f8e2 None admin] RBAC: Adding query filter params (role.id=a1464dd9ad2e4913b9d8cc6ba3c34207, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=ccf2dedaf7a449d2b51b9e9f3b420091) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:02.064859 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491b5bd-e01b-4df1-82b8-b616f586f8e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:02.066303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491b5bd-e01b-4df1-82b8-b616f586f8e2 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:02.085940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 547/1104] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:28:01 2022] GET /identity/v3/role_assignments?user.id=ccf2dedaf7a449d2b51b9e9f3b420091&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=a1464dd9ad2e4913b9d8cc6ba3c34207 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:28:02.162552 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8da07d-fbac-41a1-8dcb-e276ba4e0d63 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:02.202502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3e0dd-3fad-418d-a08b-1c3acfed912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:02.203379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3e0dd-3fad-418d-a08b-1c3acfed912c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:02.203660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3e0dd-3fad-418d-a08b-1c3acfed912c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:02.203935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3e0dd-3fad-418d-a08b-1c3acfed912c None admin] PATH_INFO: `/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:02.205163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3e0dd-3fad-418d-a08b-1c3acfed912c None admin] RBAC: Authorizing `identity:get_project(project_id=cb9f9d8a6caf48c78015a46d8b038f75)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:02.206655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3e0dd-3fad-418d-a08b-1c3acfed912c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:02.208417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 558/1105] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:28:02 2022] GET /identity/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:28:02.215333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491b5bd-e01b-4df1-82b8-b616f586f8e2 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:02.261608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:02.318249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491b5bd-e01b-4df1-82b8-b616f586f8e2 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:02.352265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bd7d0-a3c5-498b-8cd2-c210312d5d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:02.352951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bd7d0-a3c5-498b-8cd2-c210312d5d7a None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:02.353232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bd7d0-a3c5-498b-8cd2-c210312d5d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:02.353508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bd7d0-a3c5-498b-8cd2-c210312d5d7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:02.354365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bd7d0-a3c5-498b-8cd2-c210312d5d7a None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=cb9f9d8a6caf48c78015a46d8b038f75, user.id=e457edb8f0544cc782fb98f6cd407d5a) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:02.354677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bd7d0-a3c5-498b-8cd2-c210312d5d7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:02.356106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bd7d0-a3c5-498b-8cd2-c210312d5d7a None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:02.362510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 548/1106] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:28:02 2022] GET /identity/v3/role_assignments?user.id=e457edb8f0544cc782fb98f6cd407d5a&scope.project.id=cb9f9d8a6caf48c78015a46d8b038f75&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:04.386196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42f9df-656d-4225-a7af-7ee81ddfc1d0 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:04.386700 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42f9df-656d-4225-a7af-7ee81ddfc1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:04.386787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42f9df-656d-4225-a7af-7ee81ddfc1d0 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:04.388115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 559/1107] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:04.393255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abd4cf-0f7e-4d18-a0f3-c72e0d656ab3 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:04.393656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abd4cf-0f7e-4d18-a0f3-c72e0d656ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:04.393801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abd4cf-0f7e-4d18-a0f3-c72e0d656ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:04.411290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-63abd4cf-0f7e-4d18-a0f3-c72e0d656ab3 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:04.412450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-63abd4cf-0f7e-4d18-a0f3-c72e0d656ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:04.477192 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 549/1108] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:04.502461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42f9df-656d-4225-a7af-7ee81ddfc1d0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:04.505270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:04.601431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42f9df-656d-4225-a7af-7ee81ddfc1d0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:04.635453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d3c8c-c863-44c0-b683-a7a76ed016fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:04.636223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d3c8c-c863-44c0-b683-a7a76ed016fc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:04.636501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d3c8c-c863-44c0-b683-a7a76ed016fc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:04.636770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d3c8c-c863-44c0-b683-a7a76ed016fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:04.637948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d3c8c-c863-44c0-b683-a7a76ed016fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:04.639365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d3c8c-c863-44c0-b683-a7a76ed016fc None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:04.640965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 560/1109] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:28:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:04.646695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abd4cf-0f7e-4d18-a0f3-c72e0d656ab3 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:04.684503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:04.740888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abd4cf-0f7e-4d18-a0f3-c72e0d656ab3 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:04.773983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a26a2b-7b5c-49e5-b07c-4a8a76d04f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:04.774659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a26a2b-7b5c-49e5-b07c-4a8a76d04f51 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:04.774937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a26a2b-7b5c-49e5-b07c-4a8a76d04f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:04.775206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a26a2b-7b5c-49e5-b07c-4a8a76d04f51 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:04.775899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a26a2b-7b5c-49e5-b07c-4a8a76d04f51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:04.777303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a26a2b-7b5c-49e5-b07c-4a8a76d04f51 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:04.796584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 550/1110] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:28:04 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:28:06.822119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba2a5c-2bfe-4743-9955-09445be0602c None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:06.822627 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba2a5c-2bfe-4743-9955-09445be0602c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:06.822717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba2a5c-2bfe-4743-9955-09445be0602c None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:06.824051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 561/1111] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:06.829367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62c8dc-655d-41e8-962f-6f6f5d28c586 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:06.829761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62c8dc-655d-41e8-962f-6f6f5d28c586 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:06.829944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62c8dc-655d-41e8-962f-6f6f5d28c586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:06.847542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-1a62c8dc-655d-41e8-962f-6f6f5d28c586 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:06.848717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-1a62c8dc-655d-41e8-962f-6f6f5d28c586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:06.912481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 551/1112] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:06.938398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba2a5c-2bfe-4743-9955-09445be0602c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:07.007220 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:07.038413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba2a5c-2bfe-4743-9955-09445be0602c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:07.071465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b4d8e-39cf-45b1-b0e6-ffb08eb18b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:07.072356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b4d8e-39cf-45b1-b0e6-ffb08eb18b4c None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:07.072590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b4d8e-39cf-45b1-b0e6-ffb08eb18b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:07.072863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b4d8e-39cf-45b1-b0e6-ffb08eb18b4c None admin] PATH_INFO: `/v3/roles/950510c081de4ab6b12f383c0748450d` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:07.074356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b4d8e-39cf-45b1-b0e6-ffb08eb18b4c None admin] RBAC: Authorizing `identity:get_role(role_id=950510c081de4ab6b12f383c0748450d)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:07.075725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b4d8e-39cf-45b1-b0e6-ffb08eb18b4c None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:07.077090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 562/1113] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:06 2022] GET /identity/v3/roles/950510c081de4ab6b12f383c0748450d => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:07.082872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62c8dc-655d-41e8-962f-6f6f5d28c586 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:07.108079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:07.181992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62c8dc-655d-41e8-962f-6f6f5d28c586 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:07.215594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6d129-043a-4e1d-ae18-1b0c9c9b53ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:07.216445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6d129-043a-4e1d-ae18-1b0c9c9b53ce None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:07.216727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6d129-043a-4e1d-ae18-1b0c9c9b53ce None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:07.217000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6d129-043a-4e1d-ae18-1b0c9c9b53ce None admin] PATH_INFO: `/v3/users/475bcc96d6634e318e56d6905de24c23` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:07.230179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6d129-043a-4e1d-ae18-1b0c9c9b53ce None admin] RBAC: Authorizing `identity:get_user(user_id=475bcc96d6634e318e56d6905de24c23)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:07.231864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6d129-043a-4e1d-ae18-1b0c9c9b53ce None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:07.233499 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 552/1114] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:07 2022] GET /identity/v3/users/475bcc96d6634e318e56d6905de24c23 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:28:07.239254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b4d8e-39cf-45b1-b0e6-ffb08eb18b4c None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:07.333836 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b4d8e-39cf-45b1-b0e6-ffb08eb18b4c None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:07.367207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748b27e-c8f8-4f47-9ca1-ee6991a1652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:07.367864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748b27e-c8f8-4f47-9ca1-ee6991a1652b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:07.368152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748b27e-c8f8-4f47-9ca1-ee6991a1652b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:07.368432 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748b27e-c8f8-4f47-9ca1-ee6991a1652b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:07.376242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:07.433649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748b27e-c8f8-4f47-9ca1-ee6991a1652b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:07.435120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748b27e-c8f8-4f47-9ca1-ee6991a1652b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:07.529897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 563/1115] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:28:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:28:07.535653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6d129-043a-4e1d-ae18-1b0c9c9b53ce None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:07.613455 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:07.631232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6d129-043a-4e1d-ae18-1b0c9c9b53ce None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:07.664117 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b6506-ad29-4f39-81ff-e68b06f0303d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:07.665116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b6506-ad29-4f39-81ff-e68b06f0303d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:07.665399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b6506-ad29-4f39-81ff-e68b06f0303d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:07.665671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b6506-ad29-4f39-81ff-e68b06f0303d None admin] PATH_INFO: `/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:07.666862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b6506-ad29-4f39-81ff-e68b06f0303d None admin] RBAC: Authorizing `identity:get_project(project_id=cb9f9d8a6caf48c78015a46d8b038f75)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:07.668270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b6506-ad29-4f39-81ff-e68b06f0303d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:07.669951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 553/1116] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:28:07 2022] GET /identity/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:07.675915 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748b27e-c8f8-4f47-9ca1-ee6991a1652b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:07.714652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:07.773190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748b27e-c8f8-4f47-9ca1-ee6991a1652b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:07.805966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7406594-bacf-4be4-93fe-74d9fb453fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:07.806606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7406594-bacf-4be4-93fe-74d9fb453fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:07.806889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7406594-bacf-4be4-93fe-74d9fb453fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:07.807165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7406594-bacf-4be4-93fe-74d9fb453fc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:07.807987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7406594-bacf-4be4-93fe-74d9fb453fc9 None admin] RBAC: Adding query filter params (role.id=950510c081de4ab6b12f383c0748450d, scope.project.id=cb9f9d8a6caf48c78015a46d8b038f75, user.id=475bcc96d6634e318e56d6905de24c23) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:07.808300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7406594-bacf-4be4-93fe-74d9fb453fc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:07.809722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7406594-bacf-4be4-93fe-74d9fb453fc9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:07.814984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 564/1117] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:28:07 2022] GET /identity/v3/role_assignments?user.id=475bcc96d6634e318e56d6905de24c23&scope.project.id=cb9f9d8a6caf48c78015a46d8b038f75&role.id=950510c081de4ab6b12f383c0748450d => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:09.779669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558eee8-94ac-44a6-815e-b67938625001 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:09.780212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558eee8-94ac-44a6-815e-b67938625001 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:09.780212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558eee8-94ac-44a6-815e-b67938625001 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:09.781579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 554/1118] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:09.786798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f7956-f1c8-4bd5-a1b3-5ffe90c546f0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:09.787158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f7956-f1c8-4bd5-a1b3-5ffe90c546f0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:09.787358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f7956-f1c8-4bd5-a1b3-5ffe90c546f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:09.806737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2a0f7956-f1c8-4bd5-a1b3-5ffe90c546f0 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:09.807915 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2a0f7956-f1c8-4bd5-a1b3-5ffe90c546f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:09.872356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 565/1119] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:09.893812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558eee8-94ac-44a6-815e-b67938625001 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:09.994586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558eee8-94ac-44a6-815e-b67938625001 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:10.026787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c00aa-b35d-4be4-916f-a22f17f7a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:10.027649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c00aa-b35d-4be4-916f-a22f17f7a391 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:10.027929 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c00aa-b35d-4be4-916f-a22f17f7a391 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:10.028203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c00aa-b35d-4be4-916f-a22f17f7a391 None admin] PATH_INFO: `/v3/roles/950510c081de4ab6b12f383c0748450d` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:10.029653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c00aa-b35d-4be4-916f-a22f17f7a391 None admin] RBAC: Authorizing `identity:get_role(role_id=950510c081de4ab6b12f383c0748450d)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:10.031085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c00aa-b35d-4be4-916f-a22f17f7a391 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:10.032379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 555/1120] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:09 2022] GET /identity/v3/roles/950510c081de4ab6b12f383c0748450d => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:10.038051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f7956-f1c8-4bd5-a1b3-5ffe90c546f0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:10.067661 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:10.133860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f7956-f1c8-4bd5-a1b3-5ffe90c546f0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:10.166800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e44d1e-a256-4b4e-9d3a-4e85f5a2df3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:10.167667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e44d1e-a256-4b4e-9d3a-4e85f5a2df3b None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:10.167952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e44d1e-a256-4b4e-9d3a-4e85f5a2df3b None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:10.168225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e44d1e-a256-4b4e-9d3a-4e85f5a2df3b None admin] PATH_INFO: `/v3/users/475bcc96d6634e318e56d6905de24c23` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:10.169413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e44d1e-a256-4b4e-9d3a-4e85f5a2df3b None admin] RBAC: Authorizing `identity:get_user(user_id=475bcc96d6634e318e56d6905de24c23)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:10.170878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e44d1e-a256-4b4e-9d3a-4e85f5a2df3b None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:10.172518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 566/1121] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:10 2022] GET /identity/v3/users/475bcc96d6634e318e56d6905de24c23 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:10.178085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c00aa-b35d-4be4-916f-a22f17f7a391 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:10.196506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:10.274901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c00aa-b35d-4be4-916f-a22f17f7a391 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:10.307712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e13ce7-6c05-4482-b1b4-8afe9c1d0321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:10.308556 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e13ce7-6c05-4482-b1b4-8afe9c1d0321 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:10.308839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e13ce7-6c05-4482-b1b4-8afe9c1d0321 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:10.309116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e13ce7-6c05-4482-b1b4-8afe9c1d0321 None admin] PATH_INFO: `/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:10.310304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e13ce7-6c05-4482-b1b4-8afe9c1d0321 None admin] RBAC: Authorizing `identity:get_project(project_id=cb9f9d8a6caf48c78015a46d8b038f75)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:10.311716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e13ce7-6c05-4482-b1b4-8afe9c1d0321 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:10.313395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 556/1122] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:28:10 2022] GET /identity/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:10.319602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e44d1e-a256-4b4e-9d3a-4e85f5a2df3b None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:10.412918 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:10.415888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e44d1e-a256-4b4e-9d3a-4e85f5a2df3b None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:10.448724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512731-3ef3-4ad1-8fc4-d13304a17f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:10.449592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512731-3ef3-4ad1-8fc4-d13304a17f05 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:10.449861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512731-3ef3-4ad1-8fc4-d13304a17f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:10.450290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512731-3ef3-4ad1-8fc4-d13304a17f05 None admin] PATH_INFO: `/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75/users/475bcc96d6634e318e56d6905de24c23/roles/950510c081de4ab6b12f383c0748450d` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:10.452568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512731-3ef3-4ad1-8fc4-d13304a17f05 None admin] RBAC: Authorizing `identity:create_grant(project_id=cb9f9d8a6caf48c78015a46d8b038f75, user_id=475bcc96d6634e318e56d6905de24c23, role_id=950510c081de4ab6b12f383c0748450d)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:10.454095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512731-3ef3-4ad1-8fc4-d13304a17f05 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:10.467869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 567/1123] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:28:10 2022] PUT /identity/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75/users/475bcc96d6634e318e56d6905de24c23/roles/950510c081de4ab6b12f383c0748450d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:28:12.442274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969367bf-9390-4ba0-b6c9-ae895e1f72d2 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:12.442820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969367bf-9390-4ba0-b6c9-ae895e1f72d2 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:12.442820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969367bf-9390-4ba0-b6c9-ae895e1f72d2 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:12.444089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 557/1124] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:12.449460 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee75ec4-3786-440f-a99a-8dfb648d597e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:12.449868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee75ec4-3786-440f-a99a-8dfb648d597e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:12.450052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee75ec4-3786-440f-a99a-8dfb648d597e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:12.468352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7ee75ec4-3786-440f-a99a-8dfb648d597e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:12.469555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7ee75ec4-3786-440f-a99a-8dfb648d597e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:12.534475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 568/1125] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:12.557846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969367bf-9390-4ba0-b6c9-ae895e1f72d2 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:12.640659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:12.656711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969367bf-9390-4ba0-b6c9-ae895e1f72d2 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:12.688675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53a1d7-b813-4cad-923b-4bfd3bbd6ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:12.689517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53a1d7-b813-4cad-923b-4bfd3bbd6ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:12.689803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53a1d7-b813-4cad-923b-4bfd3bbd6ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:12.690114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53a1d7-b813-4cad-923b-4bfd3bbd6ef0 None admin] PATH_INFO: `/v3/roles/950510c081de4ab6b12f383c0748450d` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:12.691583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53a1d7-b813-4cad-923b-4bfd3bbd6ef0 None admin] RBAC: Authorizing `identity:get_role(role_id=950510c081de4ab6b12f383c0748450d)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:12.692990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53a1d7-b813-4cad-923b-4bfd3bbd6ef0 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:12.694345 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 558/1126] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:12 2022] GET /identity/v3/roles/950510c081de4ab6b12f383c0748450d => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:12.702010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee75ec4-3786-440f-a99a-8dfb648d597e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:12.788242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:12.799076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee75ec4-3786-440f-a99a-8dfb648d597e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:12.831712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c09454-e05c-466c-bb57-e9e3c3e6cf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:12.832564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c09454-e05c-466c-bb57-e9e3c3e6cf67 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:12.832848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c09454-e05c-466c-bb57-e9e3c3e6cf67 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:12.833125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c09454-e05c-466c-bb57-e9e3c3e6cf67 None admin] PATH_INFO: `/v3/users/475bcc96d6634e318e56d6905de24c23` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:12.834331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c09454-e05c-466c-bb57-e9e3c3e6cf67 None admin] RBAC: Authorizing `identity:get_user(user_id=475bcc96d6634e318e56d6905de24c23)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:12.835765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c09454-e05c-466c-bb57-e9e3c3e6cf67 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:12.837417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 569/1127] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:12 2022] GET /identity/v3/users/475bcc96d6634e318e56d6905de24c23 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:12.843073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53a1d7-b813-4cad-923b-4bfd3bbd6ef0 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:12.940905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53a1d7-b813-4cad-923b-4bfd3bbd6ef0 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:12.973927 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8636e79-654f-4028-bc45-06a6d0fda10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:12.974522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8636e79-654f-4028-bc45-06a6d0fda10c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:12.974807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8636e79-654f-4028-bc45-06a6d0fda10c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:12.975084 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8636e79-654f-4028-bc45-06a6d0fda10c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:12.999583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:13.042501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8636e79-654f-4028-bc45-06a6d0fda10c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:13.043971 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8636e79-654f-4028-bc45-06a6d0fda10c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:13.143500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 559/1128] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:28:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:28:13.149946 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c09454-e05c-466c-bb57-e9e3c3e6cf67 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:13.246271 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c09454-e05c-466c-bb57-e9e3c3e6cf67 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:13.279220 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba513db-9670-410e-a225-62f8d5d22ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:13.280068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba513db-9670-410e-a225-62f8d5d22ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:13.280352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba513db-9670-410e-a225-62f8d5d22ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:13.280625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba513db-9670-410e-a225-62f8d5d22ad1 None admin] PATH_INFO: `/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:13.281796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba513db-9670-410e-a225-62f8d5d22ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=cb9f9d8a6caf48c78015a46d8b038f75)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:13.283261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba513db-9670-410e-a225-62f8d5d22ad1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:13.284952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 570/1129] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:28:13 2022] GET /identity/v3/projects/cb9f9d8a6caf48c78015a46d8b038f75 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:13.291707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8636e79-654f-4028-bc45-06a6d0fda10c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:13.344441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:13.387643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8636e79-654f-4028-bc45-06a6d0fda10c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:13.419931 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81387111-b6f6-443e-aad5-e95a316a03e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:13.420565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81387111-b6f6-443e-aad5-e95a316a03e1 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:13.420844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81387111-b6f6-443e-aad5-e95a316a03e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:13.421117 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81387111-b6f6-443e-aad5-e95a316a03e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:13.421959 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81387111-b6f6-443e-aad5-e95a316a03e1 None admin] RBAC: Adding query filter params (role.id=950510c081de4ab6b12f383c0748450d, scope.project.id=cb9f9d8a6caf48c78015a46d8b038f75, user.id=475bcc96d6634e318e56d6905de24c23) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:13.422271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81387111-b6f6-443e-aad5-e95a316a03e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:13.423685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81387111-b6f6-443e-aad5-e95a316a03e1 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:13.429292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 560/1130] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:28:13 2022] GET /identity/v3/role_assignments?user.id=475bcc96d6634e318e56d6905de24c23&scope.project.id=cb9f9d8a6caf48c78015a46d8b038f75&role.id=950510c081de4ab6b12f383c0748450d => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:15.403941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0835a1-158c-45b3-9bf2-c4a8bd5832cc None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:15.404497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0835a1-158c-45b3-9bf2-c4a8bd5832cc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:15.404497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0835a1-158c-45b3-9bf2-c4a8bd5832cc None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:15.405869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 571/1131] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:15.411175 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e6e15-b0de-489f-96b7-b7d38d7551bc None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:15.411607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e6e15-b0de-489f-96b7-b7d38d7551bc None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:15.411706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e6e15-b0de-489f-96b7-b7d38d7551bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:15.429860 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-ec7e6e15-b0de-489f-96b7-b7d38d7551bc None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:15.431102 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ec7e6e15-b0de-489f-96b7-b7d38d7551bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:15.495648 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 561/1132] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:28:15.517477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0835a1-158c-45b3-9bf2-c4a8bd5832cc None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:15.544819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:15.615924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0835a1-158c-45b3-9bf2-c4a8bd5832cc None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:15.648013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a7bd9-9029-4535-8ef5-86253589d046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:15.648786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a7bd9-9029-4535-8ef5-86253589d046 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:15.649619 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a7bd9-9029-4535-8ef5-86253589d046 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:15.649891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a7bd9-9029-4535-8ef5-86253589d046 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:15.651082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a7bd9-9029-4535-8ef5-86253589d046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:15.652496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a7bd9-9029-4535-8ef5-86253589d046 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:15.654138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 572/1133] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:28:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:15.661085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e6e15-b0de-489f-96b7-b7d38d7551bc None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:15.709205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:15.757975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e6e15-b0de-489f-96b7-b7d38d7551bc None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:15.790779 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e888856-5a36-4a9b-a36f-cc1be8e2fe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:15.791433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e888856-5a36-4a9b-a36f-cc1be8e2fe9b None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:15.791714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e888856-5a36-4a9b-a36f-cc1be8e2fe9b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:15.791986 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e888856-5a36-4a9b-a36f-cc1be8e2fe9b None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:15.792694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e888856-5a36-4a9b-a36f-cc1be8e2fe9b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:15.794140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e888856-5a36-4a9b-a36f-cc1be8e2fe9b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:15.819065 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 562/1134] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:28:15 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:28:17.809947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee31d3-dac2-4ecb-a386-ee566a10ca04 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:17.810491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee31d3-dac2-4ecb-a386-ee566a10ca04 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:17.810591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee31d3-dac2-4ecb-a386-ee566a10ca04 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:17.811880 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 573/1135] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:17.817074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8647835-0843-4306-842b-f19f5c5f9ace None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:17.817452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8647835-0843-4306-842b-f19f5c5f9ace None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:17.817637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8647835-0843-4306-842b-f19f5c5f9ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:17.835886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-d8647835-0843-4306-842b-f19f5c5f9ace None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:17.837572 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d8647835-0843-4306-842b-f19f5c5f9ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:17.902523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 563/1136] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:17.926446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee31d3-dac2-4ecb-a386-ee566a10ca04 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:18.017515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:18.025631 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee31d3-dac2-4ecb-a386-ee566a10ca04 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:18.057375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc9d88-87dc-429f-9495-b7cf689c5533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:18.058187 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc9d88-87dc-429f-9495-b7cf689c5533 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:18.058465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc9d88-87dc-429f-9495-b7cf689c5533 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:18.058736 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc9d88-87dc-429f-9495-b7cf689c5533 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:18.059880 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc9d88-87dc-429f-9495-b7cf689c5533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:18.061336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc9d88-87dc-429f-9495-b7cf689c5533 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:18.062902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 574/1137] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:28:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:18.068558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8647835-0843-4306-842b-f19f5c5f9ace None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:18.098061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:18.165873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8647835-0843-4306-842b-f19f5c5f9ace None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:18.199861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937339b-407e-4ec1-b8eb-60d52f78d27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:18.200532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937339b-407e-4ec1-b8eb-60d52f78d27a None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:18.200822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937339b-407e-4ec1-b8eb-60d52f78d27a None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:18.201097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937339b-407e-4ec1-b8eb-60d52f78d27a None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:18.201799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a937339b-407e-4ec1-b8eb-60d52f78d27a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:18.203263 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a937339b-407e-4ec1-b8eb-60d52f78d27a None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:18.226236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 564/1138] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:28:18 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:28:20.259362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd2852-2675-45dd-940a-03a4c3c8a8cc None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:20.259855 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd2852-2675-45dd-940a-03a4c3c8a8cc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:20.259953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd2852-2675-45dd-940a-03a4c3c8a8cc None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:20.261331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 575/1139] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:20.266379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372500-a5fc-4f8f-a25f-03000d0ba746 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:20.266763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372500-a5fc-4f8f-a25f-03000d0ba746 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:20.266923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372500-a5fc-4f8f-a25f-03000d0ba746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:20.285073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-df372500-a5fc-4f8f-a25f-03000d0ba746 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:20.286263 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-df372500-a5fc-4f8f-a25f-03000d0ba746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:20.349225 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 565/1140] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:20.372834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd2852-2675-45dd-940a-03a4c3c8a8cc None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:20.470746 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd2852-2675-45dd-940a-03a4c3c8a8cc None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:20.506486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf99ea-dec2-4196-96f2-a2c255b91de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:20.507278 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf99ea-dec2-4196-96f2-a2c255b91de8 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:20.507564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf99ea-dec2-4196-96f2-a2c255b91de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:20.507884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf99ea-dec2-4196-96f2-a2c255b91de8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:20.517886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf99ea-dec2-4196-96f2-a2c255b91de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c5bf99ea-dec2-4196-96f2-a2c255b91de8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:28:20.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:20.527296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:28:20.528491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 576/1141] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:28:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:28:20.533666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df372500-a5fc-4f8f-a25f-03000d0ba746 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:20.551249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:20.630476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df372500-a5fc-4f8f-a25f-03000d0ba746 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:20.663049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0972b54-dfaa-41c6-948b-822d67e91940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:20.663697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0972b54-dfaa-41c6-948b-822d67e91940 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:20.663977 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0972b54-dfaa-41c6-948b-822d67e91940 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:20.664248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0972b54-dfaa-41c6-948b-822d67e91940 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:20.665532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0972b54-dfaa-41c6-948b-822d67e91940 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:20.665836 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0972b54-dfaa-41c6-948b-822d67e91940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:20.667221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0972b54-dfaa-41c6-948b-822d67e91940 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:20.675034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 566/1142] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:28:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:20.680647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf99ea-dec2-4196-96f2-a2c255b91de8 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:20.704862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:20.779039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf99ea-dec2-4196-96f2-a2c255b91de8 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:20.811206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7481c0-15a4-4176-af8d-8a3a7447ccb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:20.812065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7481c0-15a4-4176-af8d-8a3a7447ccb2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:20.812350 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7481c0-15a4-4176-af8d-8a3a7447ccb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:20.812625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7481c0-15a4-4176-af8d-8a3a7447ccb2 None admin] PATH_INFO: `/v3/users/3fc922ac87c846799cfa5fac84dae8f8` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:20.825223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7481c0-15a4-4176-af8d-8a3a7447ccb2 None admin] RBAC: Authorizing `identity:get_user(user_id=3fc922ac87c846799cfa5fac84dae8f8)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:20.826684 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7481c0-15a4-4176-af8d-8a3a7447ccb2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:20.828319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 577/1143] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:20 2022] GET /identity/v3/users/3fc922ac87c846799cfa5fac84dae8f8 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:20.833928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0972b54-dfaa-41c6-948b-822d67e91940 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:20.909564 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:20.931731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0972b54-dfaa-41c6-948b-822d67e91940 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:20.964435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fa45a-ab80-4396-a376-3297d9bae6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:20.965071 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fa45a-ab80-4396-a376-3297d9bae6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:20.965356 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fa45a-ab80-4396-a376-3297d9bae6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:20.965630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fa45a-ab80-4396-a376-3297d9bae6d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:21.035791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fa45a-ab80-4396-a376-3297d9bae6d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:21.037152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fa45a-ab80-4396-a376-3297d9bae6d1 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:21.112662 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:21.135457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 567/1144] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:28:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:28:21.141362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7481c0-15a4-4176-af8d-8a3a7447ccb2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:21.219348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:21.237859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7481c0-15a4-4176-af8d-8a3a7447ccb2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:21.270865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafef3ab-0774-4b33-a4a6-94ce665ad9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:21.271716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafef3ab-0774-4b33-a4a6-94ce665ad9eb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:21.271996 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafef3ab-0774-4b33-a4a6-94ce665ad9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:21.272266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafef3ab-0774-4b33-a4a6-94ce665ad9eb None admin] PATH_INFO: `/v3/projects/0bdb1aa7a79f45309da2e1f8ec654a4e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:21.273424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafef3ab-0774-4b33-a4a6-94ce665ad9eb None admin] RBAC: Authorizing `identity:get_project(project_id=0bdb1aa7a79f45309da2e1f8ec654a4e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:21.274877 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafef3ab-0774-4b33-a4a6-94ce665ad9eb None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:21.276596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 578/1145] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:28:21 2022] GET /identity/v3/projects/0bdb1aa7a79f45309da2e1f8ec654a4e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:21.282683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fa45a-ab80-4396-a376-3297d9bae6d1 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:21.376405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fa45a-ab80-4396-a376-3297d9bae6d1 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:21.409387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85388838-1cb9-4a5a-8408-9ebadaeb974e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:21.410052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85388838-1cb9-4a5a-8408-9ebadaeb974e None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:21.410331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85388838-1cb9-4a5a-8408-9ebadaeb974e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:21.410604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85388838-1cb9-4a5a-8408-9ebadaeb974e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:21.411422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85388838-1cb9-4a5a-8408-9ebadaeb974e None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=0bdb1aa7a79f45309da2e1f8ec654a4e, user.id=3fc922ac87c846799cfa5fac84dae8f8) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:21.411729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85388838-1cb9-4a5a-8408-9ebadaeb974e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:21.413141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85388838-1cb9-4a5a-8408-9ebadaeb974e None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:21.418433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 568/1146] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:28:21 2022] GET /identity/v3/role_assignments?user.id=3fc922ac87c846799cfa5fac84dae8f8&scope.project.id=0bdb1aa7a79f45309da2e1f8ec654a4e&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:23.365678 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1b6b4-871b-4690-9d76-9bdd212c2811 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:23.366251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1b6b4-871b-4690-9d76-9bdd212c2811 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:23.366251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1b6b4-871b-4690-9d76-9bdd212c2811 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:23.367664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 579/1147] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:23.372685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6252e-8fa2-4ad6-bd88-bb4039bff7e8 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:23.373083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6252e-8fa2-4ad6-bd88-bb4039bff7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:23.373233 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6252e-8fa2-4ad6-bd88-bb4039bff7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:23.391944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-c0a6252e-8fa2-4ad6-bd88-bb4039bff7e8 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:23.393110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c0a6252e-8fa2-4ad6-bd88-bb4039bff7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:23.427615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:23.458274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 569/1148] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:23.480559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1b6b4-871b-4690-9d76-9bdd212c2811 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:23.580174 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1b6b4-871b-4690-9d76-9bdd212c2811 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:23.612540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0a44b-2255-47d1-abd5-417f5ae1e061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:23.613314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0a44b-2255-47d1-abd5-417f5ae1e061 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:23.613594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0a44b-2255-47d1-abd5-417f5ae1e061 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:23.613866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0a44b-2255-47d1-abd5-417f5ae1e061 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:23.624091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0a44b-2255-47d1-abd5-417f5ae1e061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-35c0a44b-2255-47d1-abd5-417f5ae1e061 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:28:23.631577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:23.632670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:28:23.634447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 580/1149] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:28:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:28:23.639287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6252e-8fa2-4ad6-bd88-bb4039bff7e8 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:23.731928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6252e-8fa2-4ad6-bd88-bb4039bff7e8 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:23.764984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42f10f-fbb1-4505-87c1-40c07c4c7ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:23.765635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42f10f-fbb1-4505-87c1-40c07c4c7ade None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:23.765942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42f10f-fbb1-4505-87c1-40c07c4c7ade None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:23.766224 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42f10f-fbb1-4505-87c1-40c07c4c7ade None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:23.766965 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42f10f-fbb1-4505-87c1-40c07c4c7ade None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:23.767270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42f10f-fbb1-4505-87c1-40c07c4c7ade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:23.768621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42f10f-fbb1-4505-87c1-40c07c4c7ade None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:23.777322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 570/1150] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:28:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:23.782815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0a44b-2255-47d1-abd5-417f5ae1e061 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:23.794167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:23.879798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0a44b-2255-47d1-abd5-417f5ae1e061 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:23.911721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3bb2f-41d7-4d33-b9fb-a9c7a95381f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:23.912617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3bb2f-41d7-4d33-b9fb-a9c7a95381f0 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:23.912900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3bb2f-41d7-4d33-b9fb-a9c7a95381f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:23.913178 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3bb2f-41d7-4d33-b9fb-a9c7a95381f0 None admin] PATH_INFO: `/v3/users/3fc922ac87c846799cfa5fac84dae8f8` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:23.914912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3bb2f-41d7-4d33-b9fb-a9c7a95381f0 None admin] RBAC: Authorizing `identity:get_user(user_id=3fc922ac87c846799cfa5fac84dae8f8)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:23.916334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3bb2f-41d7-4d33-b9fb-a9c7a95381f0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:23.917968 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 581/1151] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:23 2022] GET /identity/v3/users/3fc922ac87c846799cfa5fac84dae8f8 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:23.923847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42f10f-fbb1-4505-87c1-40c07c4c7ade None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:23.957192 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:24.020874 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42f10f-fbb1-4505-87c1-40c07c4c7ade None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:24.052869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57133841-59a3-43ed-9c40-2f75e7842c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:24.053708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133841-59a3-43ed-9c40-2f75e7842c7e None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:24.054012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133841-59a3-43ed-9c40-2f75e7842c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:24.054283 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133841-59a3-43ed-9c40-2f75e7842c7e None admin] PATH_INFO: `/v3/projects/0bdb1aa7a79f45309da2e1f8ec654a4e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:24.055985 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57133841-59a3-43ed-9c40-2f75e7842c7e None admin] RBAC: Authorizing `identity:get_project(project_id=0bdb1aa7a79f45309da2e1f8ec654a4e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:24.057390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57133841-59a3-43ed-9c40-2f75e7842c7e None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:24.059106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 571/1152] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:28:23 2022] GET /identity/v3/projects/0bdb1aa7a79f45309da2e1f8ec654a4e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:24.064914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3bb2f-41d7-4d33-b9fb-a9c7a95381f0 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:24.138930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:24.161045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3bb2f-41d7-4d33-b9fb-a9c7a95381f0 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:24.194013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32063195-e760-436a-acbf-f42a2814cfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:24.194897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32063195-e760-436a-acbf-f42a2814cfc1 None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:24.195180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32063195-e760-436a-acbf-f42a2814cfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:24.195479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32063195-e760-436a-acbf-f42a2814cfc1 None admin] PATH_INFO: `/v3/projects/0bdb1aa7a79f45309da2e1f8ec654a4e/users/3fc922ac87c846799cfa5fac84dae8f8/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:24.197506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32063195-e760-436a-acbf-f42a2814cfc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bdb1aa7a79f45309da2e1f8ec654a4e, user_id=3fc922ac87c846799cfa5fac84dae8f8, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:24.198967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32063195-e760-436a-acbf-f42a2814cfc1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:24.212006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 582/1153] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:28:24 2022] PUT /identity/v3/projects/0bdb1aa7a79f45309da2e1f8ec654a4e/users/3fc922ac87c846799cfa5fac84dae8f8/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:28:26.150659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197898d3-84b8-4087-83f4-6b681746a4d3 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:26.151179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197898d3-84b8-4087-83f4-6b681746a4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:26.151179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197898d3-84b8-4087-83f4-6b681746a4d3 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:26.152549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 572/1154] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:26.157757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a57164-1ccc-42dd-bfcb-987514d88d19 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:26.158178 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a57164-1ccc-42dd-bfcb-987514d88d19 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:26.158364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a57164-1ccc-42dd-bfcb-987514d88d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:26.176167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a2a57164-1ccc-42dd-bfcb-987514d88d19 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:26.177338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a2a57164-1ccc-42dd-bfcb-987514d88d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:26.240807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 583/1155] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:26.263806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197898d3-84b8-4087-83f4-6b681746a4d3 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:26.360918 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197898d3-84b8-4087-83f4-6b681746a4d3 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:26.362439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:26.394358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfadbc6-a16b-4ac3-98dc-d3be547b7dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:26.395137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfadbc6-a16b-4ac3-98dc-d3be547b7dbf None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:26.395421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfadbc6-a16b-4ac3-98dc-d3be547b7dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:26.395700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfadbc6-a16b-4ac3-98dc-d3be547b7dbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:26.406586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfadbc6-a16b-4ac3-98dc-d3be547b7dbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-bbfadbc6-a16b-4ac3-98dc-d3be547b7dbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:28:26.414209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:26.415351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:28:26.416537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 573/1156] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:28:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:28:26.421264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a57164-1ccc-42dd-bfcb-987514d88d19 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:26.483769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:26.518871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a57164-1ccc-42dd-bfcb-987514d88d19 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:26.551165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58cb5a-334b-4a78-9280-067bcb7e508f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:26.551843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58cb5a-334b-4a78-9280-067bcb7e508f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:26.552124 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58cb5a-334b-4a78-9280-067bcb7e508f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:26.552401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58cb5a-334b-4a78-9280-067bcb7e508f None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:26.553153 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58cb5a-334b-4a78-9280-067bcb7e508f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:26.553465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58cb5a-334b-4a78-9280-067bcb7e508f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:26.555381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58cb5a-334b-4a78-9280-067bcb7e508f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:26.563604 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 584/1157] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:28:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:26.569313 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfadbc6-a16b-4ac3-98dc-d3be547b7dbf None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:26.665313 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfadbc6-a16b-4ac3-98dc-d3be547b7dbf None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:26.698413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c9139-92ef-4c58-bcd1-d27008041fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:26.699269 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c9139-92ef-4c58-bcd1-d27008041fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:26.699554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c9139-92ef-4c58-bcd1-d27008041fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:26.699832 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c9139-92ef-4c58-bcd1-d27008041fb4 None admin] PATH_INFO: `/v3/users/3fc922ac87c846799cfa5fac84dae8f8` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:26.701013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c9139-92ef-4c58-bcd1-d27008041fb4 None admin] RBAC: Authorizing `identity:get_user(user_id=3fc922ac87c846799cfa5fac84dae8f8)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:26.702467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c9139-92ef-4c58-bcd1-d27008041fb4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:26.704072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 574/1158] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:26 2022] GET /identity/v3/users/3fc922ac87c846799cfa5fac84dae8f8 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:26.709616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58cb5a-334b-4a78-9280-067bcb7e508f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:26.804268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58cb5a-334b-4a78-9280-067bcb7e508f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:26.830810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:26.838603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9610828-18cf-431a-8728-b42bb34ff507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:26.839234 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9610828-18cf-431a-8728-b42bb34ff507 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:26.839514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9610828-18cf-431a-8728-b42bb34ff507 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:26.839788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9610828-18cf-431a-8728-b42bb34ff507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:26.903457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9610828-18cf-431a-8728-b42bb34ff507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:26.904876 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9610828-18cf-431a-8728-b42bb34ff507 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:27.001199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 585/1159] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:28:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:28:27.006978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c9139-92ef-4c58-bcd1-d27008041fb4 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:27.036021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:27.103400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c9139-92ef-4c58-bcd1-d27008041fb4 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:27.137089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67426f-22a4-4a8b-96cc-86b1ea29f632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:27.137936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67426f-22a4-4a8b-96cc-86b1ea29f632 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:27.138229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67426f-22a4-4a8b-96cc-86b1ea29f632 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:27.138501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67426f-22a4-4a8b-96cc-86b1ea29f632 None admin] PATH_INFO: `/v3/projects/0bdb1aa7a79f45309da2e1f8ec654a4e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:27.139669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67426f-22a4-4a8b-96cc-86b1ea29f632 None admin] RBAC: Authorizing `identity:get_project(project_id=0bdb1aa7a79f45309da2e1f8ec654a4e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:27.141085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67426f-22a4-4a8b-96cc-86b1ea29f632 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:27.142781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 575/1160] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:28:27 2022] GET /identity/v3/projects/0bdb1aa7a79f45309da2e1f8ec654a4e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:27.148772 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9610828-18cf-431a-8728-b42bb34ff507 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:27.175842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:27.247162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9610828-18cf-431a-8728-b42bb34ff507 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:27.280166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c1e03-fe1f-456b-a432-20575edba537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:27.280765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c1e03-fe1f-456b-a432-20575edba537 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:27.281048 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c1e03-fe1f-456b-a432-20575edba537 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:27.281326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c1e03-fe1f-456b-a432-20575edba537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:27.282176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c1e03-fe1f-456b-a432-20575edba537 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=0bdb1aa7a79f45309da2e1f8ec654a4e, user.id=3fc922ac87c846799cfa5fac84dae8f8) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:27.282493 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c1e03-fe1f-456b-a432-20575edba537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:27.283924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c1e03-fe1f-456b-a432-20575edba537 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:27.289651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 586/1161] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:28:27 2022] GET /identity/v3/role_assignments?user.id=3fc922ac87c846799cfa5fac84dae8f8&scope.project.id=0bdb1aa7a79f45309da2e1f8ec654a4e&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:29.279781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37725347-b20b-48fe-b024-1a6c4c7d5e73 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:29.280312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37725347-b20b-48fe-b024-1a6c4c7d5e73 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:29.280312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37725347-b20b-48fe-b024-1a6c4c7d5e73 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:29.281666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 576/1162] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:29.286827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79c6fd-575a-4f18-88f9-6cf4d55de8fd None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:29.287238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79c6fd-575a-4f18-88f9-6cf4d55de8fd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:29.287381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79c6fd-575a-4f18-88f9-6cf4d55de8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:29.304905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6e79c6fd-575a-4f18-88f9-6cf4d55de8fd None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:29.306124 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6e79c6fd-575a-4f18-88f9-6cf4d55de8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:29.369754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 587/1163] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:29.391447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37725347-b20b-48fe-b024-1a6c4c7d5e73 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:29.485186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:29.490127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37725347-b20b-48fe-b024-1a6c4c7d5e73 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:29.523620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fae848-3a30-42ca-9de4-1d4234b9e95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:29.524256 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fae848-3a30-42ca-9de4-1d4234b9e95d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:29.524537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fae848-3a30-42ca-9de4-1d4234b9e95d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:29.524819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fae848-3a30-42ca-9de4-1d4234b9e95d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:29.588566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fae848-3a30-42ca-9de4-1d4234b9e95d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:29.590037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fae848-3a30-42ca-9de4-1d4234b9e95d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:29.678563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:29.683941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 577/1164] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:28:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:28:29.689641 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79c6fd-575a-4f18-88f9-6cf4d55de8fd None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:29.713896 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:29.788721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79c6fd-575a-4f18-88f9-6cf4d55de8fd None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:29.822416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58313316-5e27-4be7-812d-eba50bd0b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:29.823210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58313316-5e27-4be7-812d-eba50bd0b0ce None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:29.823492 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58313316-5e27-4be7-812d-eba50bd0b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:29.823769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58313316-5e27-4be7-812d-eba50bd0b0ce None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:29.830249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58313316-5e27-4be7-812d-eba50bd0b0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:29.831679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58313316-5e27-4be7-812d-eba50bd0b0ce None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-58313316-5e27-4be7-812d-eba50bd0b0ce None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:29.845463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:29.846674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:28:29.847898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 588/1165] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Jul 25 12:28:29 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:28:29.852729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fae848-3a30-42ca-9de4-1d4234b9e95d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:29.948736 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fae848-3a30-42ca-9de4-1d4234b9e95d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:29.981805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448b6e0-fb3b-4fff-adb8-a6fbf6832f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:29.982478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448b6e0-fb3b-4fff-adb8-a6fbf6832f9f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:29.982758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448b6e0-fb3b-4fff-adb8-a6fbf6832f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:29.983033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448b6e0-fb3b-4fff-adb8-a6fbf6832f9f None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:29.983750 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448b6e0-fb3b-4fff-adb8-a6fbf6832f9f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:29.984054 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448b6e0-fb3b-4fff-adb8-a6fbf6832f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:29.985399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448b6e0-fb3b-4fff-adb8-a6fbf6832f9f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:29.992379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 578/1166] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Jul 25 12:28:29 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:31.951631 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12cee2-5648-4529-b105-85c8d11c20ef None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:31.952553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12cee2-5648-4529-b105-85c8d11c20ef None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:31.952845 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12cee2-5648-4529-b105-85c8d11c20ef None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:31.954309 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 589/1167] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:31.959440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aa2e1-1dc3-4826-a59e-265f949c0cfe None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:31.959853 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aa2e1-1dc3-4826-a59e-265f949c0cfe None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:31.960003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aa2e1-1dc3-4826-a59e-265f949c0cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:31.978004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-fc2aa2e1-1dc3-4826-a59e-265f949c0cfe None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:31.979185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fc2aa2e1-1dc3-4826-a59e-265f949c0cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:32.033312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:32.043635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 579/1168] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:28:32.066154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12cee2-5648-4529-b105-85c8d11c20ef None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:32.147042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:32.166209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12cee2-5648-4529-b105-85c8d11c20ef None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:32.200870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca0344-1019-47fb-a1d9-35971ac5ffbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:32.201525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca0344-1019-47fb-a1d9-35971ac5ffbe None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:32.201861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca0344-1019-47fb-a1d9-35971ac5ffbe None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:32.202119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca0344-1019-47fb-a1d9-35971ac5ffbe None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:32.202868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca0344-1019-47fb-a1d9-35971ac5ffbe None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:32.204234 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca0344-1019-47fb-a1d9-35971ac5ffbe None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:32.219472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 590/1169] 10.222.0.60 () {60 vars in 1138 bytes} [Mon Jul 25 12:28:32 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:28:34.216018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dbfb1-83cd-4bc4-bd09-ecb15955b4b8 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:34.216490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dbfb1-83cd-4bc4-bd09-ecb15955b4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:34.216580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dbfb1-83cd-4bc4-bd09-ecb15955b4b8 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:34.217957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 580/1170] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:34.223065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3e1ee-3c74-4993-8599-1158c0631b72 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:34.223428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3e1ee-3c74-4993-8599-1158c0631b72 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:34.223628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3e1ee-3c74-4993-8599-1158c0631b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:34.241553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-cef3e1ee-3c74-4993-8599-1158c0631b72 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:34.243284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cef3e1ee-3c74-4993-8599-1158c0631b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:34.308775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 591/1171] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:34.331816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dbfb1-83cd-4bc4-bd09-ecb15955b4b8 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:34.428901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dbfb1-83cd-4bc4-bd09-ecb15955b4b8 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:34.461146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccddff-bd66-432b-8e29-b4116f161810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:34.461955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccddff-bd66-432b-8e29-b4116f161810 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:34.462235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccddff-bd66-432b-8e29-b4116f161810 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:34.462509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccddff-bd66-432b-8e29-b4116f161810 None admin] PATH_INFO: `/v3/domains/f30b1d96767347bbb34849209f4bbc2a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:34.472521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccddff-bd66-432b-8e29-b4116f161810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f30b1d96767347bbb34849209f4bbc2a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:34.473975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccddff-bd66-432b-8e29-b4116f161810 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:34.475579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 581/1172] 10.222.0.60 () {56 vars in 1183 bytes} [Mon Jul 25 12:28:34 2022] GET /identity/v3/domains/f30b1d96767347bbb34849209f4bbc2a => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:34.481970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3e1ee-3c74-4993-8599-1158c0631b72 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:34.548164 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:34.579782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3e1ee-3c74-4993-8599-1158c0631b72 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:34.613141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789c388-1804-443e-b2fc-0f4749e52660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:34.613794 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789c388-1804-443e-b2fc-0f4749e52660 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:34.614104 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789c388-1804-443e-b2fc-0f4749e52660 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:34.614382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789c388-1804-443e-b2fc-0f4749e52660 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:34.615099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789c388-1804-443e-b2fc-0f4749e52660 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:34.616523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789c388-1804-443e-b2fc-0f4749e52660 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:34.642113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 592/1173] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:28:34 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:28:36.624513 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e0d1a-349b-48db-8c64-ae5a4a81169d None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:36.625061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e0d1a-349b-48db-8c64-ae5a4a81169d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:36.625061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e0d1a-349b-48db-8c64-ae5a4a81169d None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:36.626447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 582/1174] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:36.631773 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0061010-7962-43db-9015-8c7b721a9fd4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:36.632175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0061010-7962-43db-9015-8c7b721a9fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:36.632334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0061010-7962-43db-9015-8c7b721a9fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:36.650536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c0061010-7962-43db-9015-8c7b721a9fd4 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:36.651725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c0061010-7962-43db-9015-8c7b721a9fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:36.715609 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 593/1175] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:36.740296 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e0d1a-349b-48db-8c64-ae5a4a81169d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:36.793592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:36.843069 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e0d1a-349b-48db-8c64-ae5a4a81169d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:36.876483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a898c-2138-443e-b522-6baafea07fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:36.877270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a898c-2138-443e-b522-6baafea07fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:36.877551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a898c-2138-443e-b522-6baafea07fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:36.877825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a898c-2138-443e-b522-6baafea07fb5 None admin] PATH_INFO: `/v3/domains/f30b1d96767347bbb34849209f4bbc2a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:36.879018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a898c-2138-443e-b522-6baafea07fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f30b1d96767347bbb34849209f4bbc2a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:36.880451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a898c-2138-443e-b522-6baafea07fb5 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:36.882089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 583/1176] 10.222.0.60 () {56 vars in 1183 bytes} [Mon Jul 25 12:28:36 2022] GET /identity/v3/domains/f30b1d96767347bbb34849209f4bbc2a => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:36.888406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0061010-7962-43db-9015-8c7b721a9fd4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:36.942999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:36.987317 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0061010-7962-43db-9015-8c7b721a9fd4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:37.020584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b466e-6acf-4994-80fd-2dc00a45c5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:37.021244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b466e-6acf-4994-80fd-2dc00a45c5d9 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:37.021529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b466e-6acf-4994-80fd-2dc00a45c5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:37.021808 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b466e-6acf-4994-80fd-2dc00a45c5d9 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:37.022551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b466e-6acf-4994-80fd-2dc00a45c5d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:37.024509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b466e-6acf-4994-80fd-2dc00a45c5d9 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:37.046395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 594/1177] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:28:36 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:28:39.040962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368c69f-86a6-498a-9139-4cfc18c5d3e5 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:39.041488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368c69f-86a6-498a-9139-4cfc18c5d3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:39.041942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368c69f-86a6-498a-9139-4cfc18c5d3e5 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:39.043661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 584/1178] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:39.049197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03a65c-5593-4274-a9f3-51beee4fa170 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:39.049598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03a65c-5593-4274-a9f3-51beee4fa170 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:39.049755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03a65c-5593-4274-a9f3-51beee4fa170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:39.067397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-db03a65c-5593-4274-a9f3-51beee4fa170 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:39.068574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-db03a65c-5593-4274-a9f3-51beee4fa170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:39.132320 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 595/1179] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:39.156452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368c69f-86a6-498a-9139-4cfc18c5d3e5 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:39.253467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368c69f-86a6-498a-9139-4cfc18c5d3e5 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:39.272394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:39.285756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b490efe-f4d3-454d-9059-e4996773e1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:39.286561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b490efe-f4d3-454d-9059-e4996773e1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:39.286840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b490efe-f4d3-454d-9059-e4996773e1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:39.287108 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b490efe-f4d3-454d-9059-e4996773e1f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:39.297251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b490efe-f4d3-454d-9059-e4996773e1f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-1b490efe-f4d3-454d-9059-e4996773e1f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:28:39.305140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:39.306231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:28:39.307303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 585/1180] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:28:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:28:39.312357 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03a65c-5593-4274-a9f3-51beee4fa170 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:39.352117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:39.409761 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03a65c-5593-4274-a9f3-51beee4fa170 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:39.442972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7c2a9-bbf6-48eb-87d6-a5559da10377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:39.443624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7c2a9-bbf6-48eb-87d6-a5559da10377 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:39.443906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7c2a9-bbf6-48eb-87d6-a5559da10377 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:39.444179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7c2a9-bbf6-48eb-87d6-a5559da10377 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:39.444954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7c2a9-bbf6-48eb-87d6-a5559da10377 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:39.445754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7c2a9-bbf6-48eb-87d6-a5559da10377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:39.447184 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7c2a9-bbf6-48eb-87d6-a5559da10377 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:39.455045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 596/1181] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:28:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:39.460690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b490efe-f4d3-454d-9059-e4996773e1f7 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:39.556677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b490efe-f4d3-454d-9059-e4996773e1f7 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:39.588981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75001b4-8463-489d-a569-dd27acd7ed60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:39.589830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75001b4-8463-489d-a569-dd27acd7ed60 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:39.590144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75001b4-8463-489d-a569-dd27acd7ed60 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:39.590420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75001b4-8463-489d-a569-dd27acd7ed60 None admin] PATH_INFO: `/v3/users/45c5c36cb27b4ddba5391cbaade28d15` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:39.602898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75001b4-8463-489d-a569-dd27acd7ed60 None admin] RBAC: Authorizing `identity:get_user(user_id=45c5c36cb27b4ddba5391cbaade28d15)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:39.604322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75001b4-8463-489d-a569-dd27acd7ed60 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:39.606107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 586/1182] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:39 2022] GET /identity/v3/users/45c5c36cb27b4ddba5391cbaade28d15 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:39.611847 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7c2a9-bbf6-48eb-87d6-a5559da10377 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:39.708154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7c2a9-bbf6-48eb-87d6-a5559da10377 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:39.709626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:39.741645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6d01c-0eeb-4c8f-8199-948841d0b9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:39.742840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6d01c-0eeb-4c8f-8199-948841d0b9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:39.743117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6d01c-0eeb-4c8f-8199-948841d0b9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:39.743387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6d01c-0eeb-4c8f-8199-948841d0b9b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:39.807286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6d01c-0eeb-4c8f-8199-948841d0b9b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:39.808673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6d01c-0eeb-4c8f-8199-948841d0b9b5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:39.902701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 597/1183] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:28:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:28:39.908677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75001b4-8463-489d-a569-dd27acd7ed60 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:39.948491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:40.006358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75001b4-8463-489d-a569-dd27acd7ed60 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:40.038786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f29ebe-0e62-4fc9-8016-9ae81ecc2287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:40.039627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f29ebe-0e62-4fc9-8016-9ae81ecc2287 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:40.039911 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f29ebe-0e62-4fc9-8016-9ae81ecc2287 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:40.040187 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f29ebe-0e62-4fc9-8016-9ae81ecc2287 None admin] PATH_INFO: `/v3/projects/447957a98c264f398a8565d3dd8dcdae` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:40.041353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f29ebe-0e62-4fc9-8016-9ae81ecc2287 None admin] RBAC: Authorizing `identity:get_project(project_id=447957a98c264f398a8565d3dd8dcdae)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:40.042798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f29ebe-0e62-4fc9-8016-9ae81ecc2287 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:40.044477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 587/1184] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:28:39 2022] GET /identity/v3/projects/447957a98c264f398a8565d3dd8dcdae => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:40.050499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6d01c-0eeb-4c8f-8199-948841d0b9b5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:40.052629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:40.147139 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6d01c-0eeb-4c8f-8199-948841d0b9b5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:40.178954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ec523-ebf1-4310-84d9-40aba2a3d077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:40.180105 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ec523-ebf1-4310-84d9-40aba2a3d077 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:40.180379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ec523-ebf1-4310-84d9-40aba2a3d077 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:40.180645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ec523-ebf1-4310-84d9-40aba2a3d077 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:40.181465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ec523-ebf1-4310-84d9-40aba2a3d077 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=447957a98c264f398a8565d3dd8dcdae, user.id=45c5c36cb27b4ddba5391cbaade28d15) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:40.181774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ec523-ebf1-4310-84d9-40aba2a3d077 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:40.183223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ec523-ebf1-4310-84d9-40aba2a3d077 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:40.188551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 598/1185] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:28:40 2022] GET /identity/v3/role_assignments?user.id=45c5c36cb27b4ddba5391cbaade28d15&scope.project.id=447957a98c264f398a8565d3dd8dcdae&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:42.147465 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678063c2-5151-44a4-9ad6-f67b226f4f72 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:42.148046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678063c2-5151-44a4-9ad6-f67b226f4f72 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:42.148643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678063c2-5151-44a4-9ad6-f67b226f4f72 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:42.149992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 588/1186] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:42.155198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efef31-dcf3-4d5a-bd11-7ed91e2114af None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:42.155594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efef31-dcf3-4d5a-bd11-7ed91e2114af None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:42.155776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efef31-dcf3-4d5a-bd11-7ed91e2114af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:42.173246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e6efef31-dcf3-4d5a-bd11-7ed91e2114af None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:42.174441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e6efef31-dcf3-4d5a-bd11-7ed91e2114af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:42.232763 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:42.240475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 599/1187] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:42.263559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678063c2-5151-44a4-9ad6-f67b226f4f72 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:42.360838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678063c2-5151-44a4-9ad6-f67b226f4f72 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:42.368360 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:42.395058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab68e61-dc2c-44f1-9c65-036cd6ba3278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:42.395866 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab68e61-dc2c-44f1-9c65-036cd6ba3278 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:42.396150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab68e61-dc2c-44f1-9c65-036cd6ba3278 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:42.396426 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab68e61-dc2c-44f1-9c65-036cd6ba3278 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:42.406251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab68e61-dc2c-44f1-9c65-036cd6ba3278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-aab68e61-dc2c-44f1-9c65-036cd6ba3278 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:28:42.413844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:42.414936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:28:42.415987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 589/1188] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:28:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:28:42.420993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efef31-dcf3-4d5a-bd11-7ed91e2114af None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:42.517986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efef31-dcf3-4d5a-bd11-7ed91e2114af None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:42.549701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb37a6-f777-48b5-b00b-2610934e1151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:42.550368 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb37a6-f777-48b5-b00b-2610934e1151 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:42.550651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb37a6-f777-48b5-b00b-2610934e1151 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:42.550926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb37a6-f777-48b5-b00b-2610934e1151 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:42.551665 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb37a6-f777-48b5-b00b-2610934e1151 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:42.551966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb37a6-f777-48b5-b00b-2610934e1151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:42.553325 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb37a6-f777-48b5-b00b-2610934e1151 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:42.561627 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 600/1189] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:28:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:42.567279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab68e61-dc2c-44f1-9c65-036cd6ba3278 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:42.673958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab68e61-dc2c-44f1-9c65-036cd6ba3278 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:42.706438 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167fe4b-0413-4140-b2ff-d396557c5d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:42.707284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167fe4b-0413-4140-b2ff-d396557c5d8e None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:42.707557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167fe4b-0413-4140-b2ff-d396557c5d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:42.707822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167fe4b-0413-4140-b2ff-d396557c5d8e None admin] PATH_INFO: `/v3/users/45c5c36cb27b4ddba5391cbaade28d15` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:42.708984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167fe4b-0413-4140-b2ff-d396557c5d8e None admin] RBAC: Authorizing `identity:get_user(user_id=45c5c36cb27b4ddba5391cbaade28d15)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:42.710435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167fe4b-0413-4140-b2ff-d396557c5d8e None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:42.713520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 590/1190] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:42 2022] GET /identity/v3/users/45c5c36cb27b4ddba5391cbaade28d15 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:42.719294 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb37a6-f777-48b5-b00b-2610934e1151 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:42.776684 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:42.817337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb37a6-f777-48b5-b00b-2610934e1151 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:42.848834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa167a1-08f4-41e1-af02-df5215064954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:42.849668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa167a1-08f4-41e1-af02-df5215064954 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:42.849967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa167a1-08f4-41e1-af02-df5215064954 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:42.850243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa167a1-08f4-41e1-af02-df5215064954 None admin] PATH_INFO: `/v3/projects/447957a98c264f398a8565d3dd8dcdae` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:42.851917 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa167a1-08f4-41e1-af02-df5215064954 None admin] RBAC: Authorizing `identity:get_project(project_id=447957a98c264f398a8565d3dd8dcdae)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:42.853331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa167a1-08f4-41e1-af02-df5215064954 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:42.855059 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 601/1191] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:28:42 2022] GET /identity/v3/projects/447957a98c264f398a8565d3dd8dcdae => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:42.861177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167fe4b-0413-4140-b2ff-d396557c5d8e None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:42.894836 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:42.962960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167fe4b-0413-4140-b2ff-d396557c5d8e None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:42.995706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3222c-c5a4-4ac2-bc76-6d3e9fd223c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:42.996580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3222c-c5a4-4ac2-bc76-6d3e9fd223c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:42.996858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3222c-c5a4-4ac2-bc76-6d3e9fd223c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:42.997125 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3222c-c5a4-4ac2-bc76-6d3e9fd223c8 None admin] PATH_INFO: `/v3/projects/447957a98c264f398a8565d3dd8dcdae/users/45c5c36cb27b4ddba5391cbaade28d15/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:42.999059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3222c-c5a4-4ac2-bc76-6d3e9fd223c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=447957a98c264f398a8565d3dd8dcdae, user_id=45c5c36cb27b4ddba5391cbaade28d15, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:43.000464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3222c-c5a4-4ac2-bc76-6d3e9fd223c8 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:43.012642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 591/1192] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:28:42 2022] PUT /identity/v3/projects/447957a98c264f398a8565d3dd8dcdae/users/45c5c36cb27b4ddba5391cbaade28d15/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:28:44.986942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c18c73-d77f-4a02-b9da-2e65036a8c4f None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:44.987490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c18c73-d77f-4a02-b9da-2e65036a8c4f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:44.987490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c18c73-d77f-4a02-b9da-2e65036a8c4f None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:44.988846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 602/1193] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:28:44.994667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e4161-dd79-40ca-850a-cd9dd21d21b5 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:44.995081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e4161-dd79-40ca-850a-cd9dd21d21b5 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:44.995228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e4161-dd79-40ca-850a-cd9dd21d21b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:45.012931 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-7f8e4161-dd79-40ca-850a-cd9dd21d21b5 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:28:45.014137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7f8e4161-dd79-40ca-850a-cd9dd21d21b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:45.078677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 592/1194] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:28:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:28:45.100668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c18c73-d77f-4a02-b9da-2e65036a8c4f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:45.203591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c18c73-d77f-4a02-b9da-2e65036a8c4f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:45.228491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:45.237479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb9f1b-cc9a-4bf4-8b4d-2d29faeba404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:45.238235 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb9f1b-cc9a-4bf4-8b4d-2d29faeba404 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:45.238517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb9f1b-cc9a-4bf4-8b4d-2d29faeba404 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:45.238789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb9f1b-cc9a-4bf4-8b4d-2d29faeba404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:45.249165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb9f1b-cc9a-4bf4-8b4d-2d29faeba404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-04eb9f1b-cc9a-4bf4-8b4d-2d29faeba404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:28:45.256791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:28:45.257948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:28:45.259054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 603/1195] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:28:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:28:45.264056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e4161-dd79-40ca-850a-cd9dd21d21b5 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:45.288107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:45.361994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e4161-dd79-40ca-850a-cd9dd21d21b5 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:45.393933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800ebea-8ba9-45d5-be9e-02841d96888c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:45.394608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800ebea-8ba9-45d5-be9e-02841d96888c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:45.394892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800ebea-8ba9-45d5-be9e-02841d96888c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:45.395169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800ebea-8ba9-45d5-be9e-02841d96888c None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:45.395913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800ebea-8ba9-45d5-be9e-02841d96888c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:45.396222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800ebea-8ba9-45d5-be9e-02841d96888c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:45.397581 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800ebea-8ba9-45d5-be9e-02841d96888c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:45.405877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 593/1196] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:28:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:28:45.411495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb9f1b-cc9a-4bf4-8b4d-2d29faeba404 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:45.514180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb9f1b-cc9a-4bf4-8b4d-2d29faeba404 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:45.547130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982abff-172f-4583-ae8f-35b4e98d9bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:45.547981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982abff-172f-4583-ae8f-35b4e98d9bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:45.548263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982abff-172f-4583-ae8f-35b4e98d9bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:45.548538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982abff-172f-4583-ae8f-35b4e98d9bf2 None admin] PATH_INFO: `/v3/users/45c5c36cb27b4ddba5391cbaade28d15` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:45.549702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982abff-172f-4583-ae8f-35b4e98d9bf2 None admin] RBAC: Authorizing `identity:get_user(user_id=45c5c36cb27b4ddba5391cbaade28d15)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:45.551164 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982abff-172f-4583-ae8f-35b4e98d9bf2 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:45.552783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 604/1197] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Jul 25 12:28:45 2022] GET /identity/v3/users/45c5c36cb27b4ddba5391cbaade28d15 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:45.558595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800ebea-8ba9-45d5-be9e-02841d96888c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:45.639443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:45.654016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800ebea-8ba9-45d5-be9e-02841d96888c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:45.684906 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcd325-6e4c-48db-b96a-2323c6cfbd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:45.685532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcd325-6e4c-48db-b96a-2323c6cfbd3d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:45.685811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcd325-6e4c-48db-b96a-2323c6cfbd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:45.686116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcd325-6e4c-48db-b96a-2323c6cfbd3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:45.749230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bcd325-6e4c-48db-b96a-2323c6cfbd3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:45.750673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bcd325-6e4c-48db-b96a-2323c6cfbd3d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:45.831017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:45.846569 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 594/1198] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:28:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:28:45.852362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982abff-172f-4583-ae8f-35b4e98d9bf2 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:45.905687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:45.948451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982abff-172f-4583-ae8f-35b4e98d9bf2 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:45.980540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fe077-384a-4275-9614-3b9a35e32bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:45.981637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fe077-384a-4275-9614-3b9a35e32bcb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:45.981939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fe077-384a-4275-9614-3b9a35e32bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:45.982227 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fe077-384a-4275-9614-3b9a35e32bcb None admin] PATH_INFO: `/v3/projects/447957a98c264f398a8565d3dd8dcdae` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:45.983392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33fe077-384a-4275-9614-3b9a35e32bcb None admin] RBAC: Authorizing `identity:get_project(project_id=447957a98c264f398a8565d3dd8dcdae)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:45.984813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33fe077-384a-4275-9614-3b9a35e32bcb None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:45.986527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 605/1199] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Jul 25 12:28:45 2022] GET /identity/v3/projects/447957a98c264f398a8565d3dd8dcdae => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:45.993014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcd325-6e4c-48db-b96a-2323c6cfbd3d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:46.087454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcd325-6e4c-48db-b96a-2323c6cfbd3d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:46.119706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923abec-489e-42cd-899e-568e8d6c175d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:46.120337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923abec-489e-42cd-899e-568e8d6c175d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:46.120615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923abec-489e-42cd-899e-568e8d6c175d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:46.120886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923abec-489e-42cd-899e-568e8d6c175d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:46.121702 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923abec-489e-42cd-899e-568e8d6c175d None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=447957a98c264f398a8565d3dd8dcdae, user.id=45c5c36cb27b4ddba5391cbaade28d15) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:28:46.122042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923abec-489e-42cd-899e-568e8d6c175d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:46.123453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923abec-489e-42cd-899e-568e8d6c175d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:46.128979 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 595/1200] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:28:45 2022] GET /identity/v3/role_assignments?user.id=45c5c36cb27b4ddba5391cbaade28d15&scope.project.id=447957a98c264f398a8565d3dd8dcdae&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:28:55.976649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=39 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:56.126437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=75 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:40.734819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a56e47-1348-456c-bd29-4baa03910292 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:40.735441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a56e47-1348-456c-bd29-4baa03910292 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:40.735441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a56e47-1348-456c-bd29-4baa03910292 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:40.736953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 606/1201] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:29:40.745447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7b79-6cea-4a07-b69d-4391ec4c6cbd None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:40.746073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7b79-6cea-4a07-b69d-4391ec4c6cbd None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:40.746073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7b79-6cea-4a07-b69d-4391ec4c6cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:40.766260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f98b7b79-6cea-4a07-b69d-4391ec4c6cbd None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:29:40.767465 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f98b7b79-6cea-4a07-b69d-4391ec4c6cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:40.850799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 596/1202] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:29:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:29:40.880999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a56e47-1348-456c-bd29-4baa03910292 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:41.022914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a56e47-1348-456c-bd29-4baa03910292 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:41.066751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db5a88-e071-405d-a230-dc43fe2c656d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:41.067553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db5a88-e071-405d-a230-dc43fe2c656d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:41.067910 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db5a88-e071-405d-a230-dc43fe2c656d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:41.068214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db5a88-e071-405d-a230-dc43fe2c656d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:41.087304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db5a88-e071-405d-a230-dc43fe2c656d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:41.088884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db5a88-e071-405d-a230-dc43fe2c656d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-c4db5a88-e071-405d-a230-dc43fe2c656d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:29:41.112965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:41.114029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:41.115582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:29:41.115582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:29:41.115582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:41.115582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:29:41.118072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 607/1203] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:41.123299 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b7b79-6cea-4a07-b69d-4391ec4c6cbd None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:41.265594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b7b79-6cea-4a07-b69d-4391ec4c6cbd None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:41.301439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:41.310863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b087c87-a33d-4251-a275-2358ce4a6516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:41.311579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b087c87-a33d-4251-a275-2358ce4a6516 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:41.311865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b087c87-a33d-4251-a275-2358ce4a6516 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:41.312148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b087c87-a33d-4251-a275-2358ce4a6516 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:41.312948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b087c87-a33d-4251-a275-2358ce4a6516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:41.313217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b087c87-a33d-4251-a275-2358ce4a6516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:41.314725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b087c87-a33d-4251-a275-2358ce4a6516 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:41.327088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 597/1204] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:41.333180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db5a88-e071-405d-a230-dc43fe2c656d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:41.403002 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:41.469511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db5a88-e071-405d-a230-dc43fe2c656d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:41.513304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b640f3-5e4a-489e-81db-0a41fcbbfe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:41.514022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b640f3-5e4a-489e-81db-0a41fcbbfe67 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:41.514315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b640f3-5e4a-489e-81db-0a41fcbbfe67 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:41.514595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b640f3-5e4a-489e-81db-0a41fcbbfe67 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:41.515304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b640f3-5e4a-489e-81db-0a41fcbbfe67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:41.516763 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b640f3-5e4a-489e-81db-0a41fcbbfe67 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:41.543155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 608/1205] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Jul 25 12:29:41 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:29:43.786850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d67c77-025d-4bd0-b196-4c8a8f4b7205 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:43.787428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d67c77-025d-4bd0-b196-4c8a8f4b7205 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:43.787428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d67c77-025d-4bd0-b196-4c8a8f4b7205 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:43.788767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 598/1206] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:29:43.794900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3371c12-25a6-41e1-b2c1-d2db3b8b2040 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:43.795396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3371c12-25a6-41e1-b2c1-d2db3b8b2040 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:43.795546 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3371c12-25a6-41e1-b2c1-d2db3b8b2040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:43.820844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a3371c12-25a6-41e1-b2c1-d2db3b8b2040 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:29:43.822256 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a3371c12-25a6-41e1-b2c1-d2db3b8b2040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:43.909946 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 609/1207] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:29:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:29:43.934826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d67c77-025d-4bd0-b196-4c8a8f4b7205 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:44.055972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d67c77-025d-4bd0-b196-4c8a8f4b7205 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:44.100444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d42dab-b661-4de6-82db-1b79bb05d64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:44.101618 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d42dab-b661-4de6-82db-1b79bb05d64f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:44.102096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d42dab-b661-4de6-82db-1b79bb05d64f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:44.102531 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d42dab-b661-4de6-82db-1b79bb05d64f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:44.115254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d42dab-b661-4de6-82db-1b79bb05d64f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-d8d42dab-b661-4de6-82db-1b79bb05d64f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:29:44.123679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:44.124817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:29:44.131524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 599/1208] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:29:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:44.132433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3371c12-25a6-41e1-b2c1-d2db3b8b2040 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:44.183379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:44.260518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3371c12-25a6-41e1-b2c1-d2db3b8b2040 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:44.302238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f7e76-b20d-49a5-9a93-f226093e8006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:44.303073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f7e76-b20d-49a5-9a93-f226093e8006 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:44.303365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f7e76-b20d-49a5-9a93-f226093e8006 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:44.303593 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f7e76-b20d-49a5-9a93-f226093e8006 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:44.304430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f7e76-b20d-49a5-9a93-f226093e8006 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:44.304767 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f7e76-b20d-49a5-9a93-f226093e8006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:44.306289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f7e76-b20d-49a5-9a93-f226093e8006 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:44.315189 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 610/1209] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:29:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:29:44.321215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d42dab-b661-4de6-82db-1b79bb05d64f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:44.381565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:44.437365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d42dab-b661-4de6-82db-1b79bb05d64f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:44.477750 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232ac30-9575-4829-a18b-52f63fb9727b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:44.478581 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232ac30-9575-4829-a18b-52f63fb9727b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:44.478907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232ac30-9575-4829-a18b-52f63fb9727b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:44.479186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232ac30-9575-4829-a18b-52f63fb9727b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:44.491330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232ac30-9575-4829-a18b-52f63fb9727b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:44.492850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232ac30-9575-4829-a18b-52f63fb9727b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-9232ac30-9575-4829-a18b-52f63fb9727b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:29:44.512718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:44.514085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:44.514991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:29:44.514991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:29:44.514991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:44.514991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:29:44.514991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 600/1210] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:44.520541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f7e76-b20d-49a5-9a93-f226093e8006 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:44.643022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:44.646812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f7e76-b20d-49a5-9a93-f226093e8006 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:44.688518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14500b-ff47-49f6-9cde-0271cfd727c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:44.689170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14500b-ff47-49f6-9cde-0271cfd727c1 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:44.689547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14500b-ff47-49f6-9cde-0271cfd727c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:44.689827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14500b-ff47-49f6-9cde-0271cfd727c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:44.690600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14500b-ff47-49f6-9cde-0271cfd727c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:44.690895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14500b-ff47-49f6-9cde-0271cfd727c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:44.692376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14500b-ff47-49f6-9cde-0271cfd727c1 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:44.705140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 611/1211] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:44.711724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232ac30-9575-4829-a18b-52f63fb9727b None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:44.838224 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232ac30-9575-4829-a18b-52f63fb9727b None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:44.840972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:44.883258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7516b-7189-43f4-ae0d-0625b512af41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:44.884175 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7516b-7189-43f4-ae0d-0625b512af41 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:44.884456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7516b-7189-43f4-ae0d-0625b512af41 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:44.884732 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7516b-7189-43f4-ae0d-0625b512af41 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:44.892769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7516b-7189-43f4-ae0d-0625b512af41 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:44.894744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7516b-7189-43f4-ae0d-0625b512af41 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-26f7516b-7189-43f4-ae0d-0625b512af41 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:29:44.911249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:44.912751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:44.913701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:44.913701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:44.913701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:29:44.913701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:29:44.913701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:29:44.913701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:29:44.913701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:29:44.913701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:29:44.913701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:29:44.913701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:29:44.914292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 601/1212] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:44.922305 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14500b-ff47-49f6-9cde-0271cfd727c1 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:45.044398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14500b-ff47-49f6-9cde-0271cfd727c1 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:45.082398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c140e-21f4-45b2-921c-e3230b8e41e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:45.083175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c140e-21f4-45b2-921c-e3230b8e41e3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:45.083427 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c140e-21f4-45b2-921c-e3230b8e41e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:45.083689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c140e-21f4-45b2-921c-e3230b8e41e3 None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:45.084577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c140e-21f4-45b2-921c-e3230b8e41e3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:45.084883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c140e-21f4-45b2-921c-e3230b8e41e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:45.086362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c140e-21f4-45b2-921c-e3230b8e41e3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:45.110975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 612/1213] 10.222.0.60 () {56 vars in 1143 bytes} [Mon Jul 25 12:29:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:45.119590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7516b-7189-43f4-ae0d-0625b512af41 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:45.246020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7516b-7189-43f4-ae0d-0625b512af41 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:45.284968 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c5c99-b8f9-4e76-b6cc-2d693a214143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:45.285717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c5c99-b8f9-4e76-b6cc-2d693a214143 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:45.286100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c5c99-b8f9-4e76-b6cc-2d693a214143 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:45.286996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c5c99-b8f9-4e76-b6cc-2d693a214143 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:45.319189 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:45.361338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c5c99-b8f9-4e76-b6cc-2d693a214143 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:45.363006 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c5c99-b8f9-4e76-b6cc-2d693a214143 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:45.495440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 602/1214] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:29:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:29:45.496677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c140e-21f4-45b2-921c-e3230b8e41e3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:45.518083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:45.611210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c140e-21f4-45b2-921c-e3230b8e41e3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:45.654516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008064be-e8bd-422b-930a-c131eb31f966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:45.655540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008064be-e8bd-422b-930a-c131eb31f966 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:45.655958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008064be-e8bd-422b-930a-c131eb31f966 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:45.656228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008064be-e8bd-422b-930a-c131eb31f966 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:45.668928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008064be-e8bd-422b-930a-c131eb31f966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:45.670439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008064be-e8bd-422b-930a-c131eb31f966 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-008064be-e8bd-422b-930a-c131eb31f966 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:29:45.693395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:45.694443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:45.700207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:29:45.700207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:29:45.700207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:45.700207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:29:45.700207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 613/1215] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:45.700658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c5c99-b8f9-4e76-b6cc-2d693a214143 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:45.775164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:45.817466 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c5c99-b8f9-4e76-b6cc-2d693a214143 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:45.852137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43acda-96a2-4332-9c36-c222a3ee036c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:45.852798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43acda-96a2-4332-9c36-c222a3ee036c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:45.853068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43acda-96a2-4332-9c36-c222a3ee036c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:45.853337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43acda-96a2-4332-9c36-c222a3ee036c None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:45.854079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d43acda-96a2-4332-9c36-c222a3ee036c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:45.854382 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d43acda-96a2-4332-9c36-c222a3ee036c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:45.855735 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d43acda-96a2-4332-9c36-c222a3ee036c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:45.872494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 603/1216] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:45.873599 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008064be-e8bd-422b-930a-c131eb31f966 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:45.948053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:45.991978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008064be-e8bd-422b-930a-c131eb31f966 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:46.038869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30501d-713d-4f45-9815-921b50343601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:46.039792 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30501d-713d-4f45-9815-921b50343601 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:46.040088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30501d-713d-4f45-9815-921b50343601 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:46.040410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30501d-713d-4f45-9815-921b50343601 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:46.048044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30501d-713d-4f45-9815-921b50343601 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:46.049656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30501d-713d-4f45-9815-921b50343601 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-fc30501d-713d-4f45-9815-921b50343601 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:29:46.066264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:46.067728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:46.069218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:29:46.069218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:29:46.069218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:29:46.069218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:29:46.069218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:29:46.069218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:29:46.069218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:29:46.069218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:29:46.069218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 614/1217] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:29:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:46.074551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43acda-96a2-4332-9c36-c222a3ee036c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:46.196039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43acda-96a2-4332-9c36-c222a3ee036c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:46.219147 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:46.233450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e11e0-26b5-48a8-a4d9-15937309d581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:46.234154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e11e0-26b5-48a8-a4d9-15937309d581 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:46.234477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e11e0-26b5-48a8-a4d9-15937309d581 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:46.234758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e11e0-26b5-48a8-a4d9-15937309d581 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:46.235545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e11e0-26b5-48a8-a4d9-15937309d581 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:46.235853 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e11e0-26b5-48a8-a4d9-15937309d581 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:46.237291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e11e0-26b5-48a8-a4d9-15937309d581 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:46.251180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 604/1218] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:29:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:46.257940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30501d-713d-4f45-9815-921b50343601 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:46.372412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30501d-713d-4f45-9815-921b50343601 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:46.409461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:46.417225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4d26b-f1d3-4863-b0d7-50ae7012ff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:46.418075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4d26b-f1d3-4863-b0d7-50ae7012ff40 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:46.418404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4d26b-f1d3-4863-b0d7-50ae7012ff40 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:46.418685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4d26b-f1d3-4863-b0d7-50ae7012ff40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:46.419533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4d26b-f1d3-4863-b0d7-50ae7012ff40 None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=81037f61e01e4a93982503720f7404e3) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:46.419850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4d26b-f1d3-4863-b0d7-50ae7012ff40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:46.421295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4d26b-f1d3-4863-b0d7-50ae7012ff40 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:46.429954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 615/1219] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:29:46 2022] GET /identity/v3/role_assignments?user.id=81037f61e01e4a93982503720f7404e3&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:48.552607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd8f91-1d2d-4f7d-be0b-ca8e1560f9c5 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:48.553143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd8f91-1d2d-4f7d-be0b-ca8e1560f9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:48.553143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd8f91-1d2d-4f7d-be0b-ca8e1560f9c5 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:48.554555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 605/1220] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:29:48.559709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f6a8e-5c70-4d7c-bcd5-d2afdf7ef0ea None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:48.560157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f6a8e-5c70-4d7c-bcd5-d2afdf7ef0ea None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:48.560249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f6a8e-5c70-4d7c-bcd5-d2afdf7ef0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:48.578064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-539f6a8e-5c70-4d7c-bcd5-d2afdf7ef0ea None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:29:48.579332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-539f6a8e-5c70-4d7c-bcd5-d2afdf7ef0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:48.658181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 616/1221] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:29:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:29:48.685041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd8f91-1d2d-4f7d-be0b-ca8e1560f9c5 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:48.798702 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd8f91-1d2d-4f7d-be0b-ca8e1560f9c5 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:48.832335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b9b18-2be0-458a-b06e-df1d19032f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:48.833638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b9b18-2be0-458a-b06e-df1d19032f02 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:48.833956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b9b18-2be0-458a-b06e-df1d19032f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:48.834230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b9b18-2be0-458a-b06e-df1d19032f02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:48.844534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b9b18-2be0-458a-b06e-df1d19032f02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-300b9b18-2be0-458a-b06e-df1d19032f02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:29:48.852792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:48.853765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:29:48.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 606/1222] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:29:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:48.860596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f6a8e-5c70-4d7c-bcd5-d2afdf7ef0ea None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:48.986360 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f6a8e-5c70-4d7c-bcd5-d2afdf7ef0ea None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:49.014588 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:49.026353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c4cd0-ae53-40ad-a994-990993a8a605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:49.027027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c4cd0-ae53-40ad-a994-990993a8a605 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:49.027311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c4cd0-ae53-40ad-a994-990993a8a605 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:49.027591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c4cd0-ae53-40ad-a994-990993a8a605 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:49.028337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c4cd0-ae53-40ad-a994-990993a8a605 None admin] RBAC: Adding query filter params (name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:49.028647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c4cd0-ae53-40ad-a994-990993a8a605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:49.030050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c4cd0-ae53-40ad-a994-990993a8a605 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:49.043972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 617/1223] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:29:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:49.050059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b9b18-2be0-458a-b06e-df1d19032f02 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:49.095864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:49.157262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b9b18-2be0-458a-b06e-df1d19032f02 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:49.196703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb69f15-95e8-4a39-8bf4-32eca2cecc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:49.197501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb69f15-95e8-4a39-8bf4-32eca2cecc1f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:49.197782 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb69f15-95e8-4a39-8bf4-32eca2cecc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:49.198085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb69f15-95e8-4a39-8bf4-32eca2cecc1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:49.209092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb69f15-95e8-4a39-8bf4-32eca2cecc1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:49.210588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb69f15-95e8-4a39-8bf4-32eca2cecc1f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-1cb69f15-95e8-4a39-8bf4-32eca2cecc1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:29:49.228156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:49.229231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:49.230329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:29:49.230329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:29:49.230329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:49.230329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:29:49.230329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 607/1224] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:49.235750 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c4cd0-ae53-40ad-a994-990993a8a605 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:49.338751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c4cd0-ae53-40ad-a994-990993a8a605 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:49.372861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438452be-f0ce-48f8-a5b7-d392319f8c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:49.373457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438452be-f0ce-48f8-a5b7-d392319f8c11 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:49.373743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438452be-f0ce-48f8-a5b7-d392319f8c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:49.374058 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438452be-f0ce-48f8-a5b7-d392319f8c11 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:49.374786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438452be-f0ce-48f8-a5b7-d392319f8c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:49.375094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438452be-f0ce-48f8-a5b7-d392319f8c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:49.376477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438452be-f0ce-48f8-a5b7-d392319f8c11 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:49.389438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 618/1225] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:49.395003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb69f15-95e8-4a39-8bf4-32eca2cecc1f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:49.494004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:49.503699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb69f15-95e8-4a39-8bf4-32eca2cecc1f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:49.541650 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1a5f7-ebf5-4af5-a3d9-74414ec6ce94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:49.542544 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1a5f7-ebf5-4af5-a3d9-74414ec6ce94 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:49.542868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1a5f7-ebf5-4af5-a3d9-74414ec6ce94 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:49.543154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1a5f7-ebf5-4af5-a3d9-74414ec6ce94 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:49.551063 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1a5f7-ebf5-4af5-a3d9-74414ec6ce94 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:49.552586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1a5f7-ebf5-4af5-a3d9-74414ec6ce94 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-fbf1a5f7-ebf5-4af5-a3d9-74414ec6ce94 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:29:49.567121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:49.569001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:49.569976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:49.569976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:49.569976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:29:49.569976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:29:49.569976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:29:49.569976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:29:49.569976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:29:49.569976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:29:49.569976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:29:49.569976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:29:49.569976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 608/1226] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:29:49.575059 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438452be-f0ce-48f8-a5b7-d392319f8c11 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:49.627928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:49.689174 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438452be-f0ce-48f8-a5b7-d392319f8c11 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:49.726902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d6612-c512-4860-ace3-9772af45829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:49.727609 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d6612-c512-4860-ace3-9772af45829e None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:49.727853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d6612-c512-4860-ace3-9772af45829e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:49.728124 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d6612-c512-4860-ace3-9772af45829e None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:49.728996 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d6612-c512-4860-ace3-9772af45829e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:49.729305 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d6612-c512-4860-ace3-9772af45829e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:49.731142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d6612-c512-4860-ace3-9772af45829e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:49.755711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 619/1227] 10.222.0.60 () {56 vars in 1143 bytes} [Mon Jul 25 12:29:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:49.756979 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1a5f7-ebf5-4af5-a3d9-74414ec6ce94 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:49.862033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1a5f7-ebf5-4af5-a3d9-74414ec6ce94 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:49.899395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc1d0b-6e29-441f-b0b9-e7daa8a1848b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:49.900119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc1d0b-6e29-441f-b0b9-e7daa8a1848b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:49.900401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc1d0b-6e29-441f-b0b9-e7daa8a1848b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:49.900679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc1d0b-6e29-441f-b0b9-e7daa8a1848b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:49.912615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc1d0b-6e29-441f-b0b9-e7daa8a1848b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:49.914113 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc1d0b-6e29-441f-b0b9-e7daa8a1848b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-37cc1d0b-6e29-441f-b0b9-e7daa8a1848b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:29:49.932645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:49.933804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:49.934796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:29:49.934796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:29:49.934796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:49.934796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:29:49.934796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 609/1228] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:49.939831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d6612-c512-4860-ace3-9772af45829e None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:50.065678 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d6612-c512-4860-ace3-9772af45829e None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:50.069240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:50.103989 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578f771-7a36-4288-b042-4e108ff76c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:50.104664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578f771-7a36-4288-b042-4e108ff76c98 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:50.104966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578f771-7a36-4288-b042-4e108ff76c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:50.105250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578f771-7a36-4288-b042-4e108ff76c98 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:50.106227 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578f771-7a36-4288-b042-4e108ff76c98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:50.106532 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578f771-7a36-4288-b042-4e108ff76c98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:50.107908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578f771-7a36-4288-b042-4e108ff76c98 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:50.120002 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 620/1229] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:50.127231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc1d0b-6e29-441f-b0b9-e7daa8a1848b None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:50.137590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:50.235915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc1d0b-6e29-441f-b0b9-e7daa8a1848b None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:50.276527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d3c38-989c-49c4-a4be-13f607e81d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:50.277515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d3c38-989c-49c4-a4be-13f607e81d3f None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:50.278001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d3c38-989c-49c4-a4be-13f607e81d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:50.278525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d3c38-989c-49c4-a4be-13f607e81d3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:50.288223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d3c38-989c-49c4-a4be-13f607e81d3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:50.289647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d3c38-989c-49c4-a4be-13f607e81d3f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-605d3c38-989c-49c4-a4be-13f607e81d3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:29:50.304978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:50.306119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:50.307479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:29:50.307479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:29:50.307479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:29:50.307479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:29:50.307479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:29:50.307479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:29:50.307479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:29:50.307479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:29:50.307479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 610/1230] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:29:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:50.312420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578f771-7a36-4288-b042-4e108ff76c98 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:50.420020 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578f771-7a36-4288-b042-4e108ff76c98 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:50.454603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9486702-8120-4161-b6a3-cfb936c49ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:50.455301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9486702-8120-4161-b6a3-cfb936c49ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:50.455576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9486702-8120-4161-b6a3-cfb936c49ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:50.455925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9486702-8120-4161-b6a3-cfb936c49ea5 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:50.456698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9486702-8120-4161-b6a3-cfb936c49ea5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:50.457007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9486702-8120-4161-b6a3-cfb936c49ea5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:50.458525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9486702-8120-4161-b6a3-cfb936c49ea5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:50.470575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 621/1231] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:29:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:50.477042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d3c38-989c-49c4-a4be-13f607e81d3f None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:50.551189 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:50.590318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d3c38-989c-49c4-a4be-13f607e81d3f None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:50.630882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96468433-da14-4845-992b-d02ee1a6414f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:50.634536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468433-da14-4845-992b-d02ee1a6414f None admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:50.634807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468433-da14-4845-992b-d02ee1a6414f None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:50.635142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468433-da14-4845-992b-d02ee1a6414f None admin] PATH_INFO: `/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/81037f61e01e4a93982503720f7404e3/roles/d9c5c1adb796486d83edcd6e728a5de9` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:50.651258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96468433-da14-4845-992b-d02ee1a6414f None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff5a41f8ca6438295c512792d2022e4, user_id=81037f61e01e4a93982503720f7404e3, role_id=d9c5c1adb796486d83edcd6e728a5de9)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:50.652817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96468433-da14-4845-992b-d02ee1a6414f None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:50.668169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 611/1232] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:29:50 2022] PUT /identity/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/81037f61e01e4a93982503720f7404e3/roles/d9c5c1adb796486d83edcd6e728a5de9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:29:52.818374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d068970-6b34-4617-9937-aa98a0aec714 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:52.818929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d068970-6b34-4617-9937-aa98a0aec714 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:52.818929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d068970-6b34-4617-9937-aa98a0aec714 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:52.820299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 622/1233] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:29:52.826217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b295af-fcde-40b7-a0f9-5b4a6ac7e3c9 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:52.826731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b295af-fcde-40b7-a0f9-5b4a6ac7e3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:52.826809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b295af-fcde-40b7-a0f9-5b4a6ac7e3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:52.845010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-96b295af-fcde-40b7-a0f9-5b4a6ac7e3c9 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:29:52.846397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-96b295af-fcde-40b7-a0f9-5b4a6ac7e3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:52.919722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 612/1234] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:29:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:29:52.945069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d068970-6b34-4617-9937-aa98a0aec714 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:52.982255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:53.066255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d068970-6b34-4617-9937-aa98a0aec714 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:53.104109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4e46f-c46f-443e-ae07-e295c7f9b1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:53.105040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4e46f-c46f-443e-ae07-e295c7f9b1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:53.105460 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4e46f-c46f-443e-ae07-e295c7f9b1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:53.106070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4e46f-c46f-443e-ae07-e295c7f9b1f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:53.118086 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4e46f-c46f-443e-ae07-e295c7f9b1f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-75d4e46f-c46f-443e-ae07-e295c7f9b1f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:29:53.126382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:53.127622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:29:53.128454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 623/1235] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Jul 25 12:29:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:53.134853 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b295af-fcde-40b7-a0f9-5b4a6ac7e3c9 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:53.185890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:53.242471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b295af-fcde-40b7-a0f9-5b4a6ac7e3c9 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:53.278150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc767b6-cfc2-4a02-9a22-044f2fbca6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:53.278823 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc767b6-cfc2-4a02-9a22-044f2fbca6fb None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:53.279111 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc767b6-cfc2-4a02-9a22-044f2fbca6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:53.279393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc767b6-cfc2-4a02-9a22-044f2fbca6fb None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:53.280142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc767b6-cfc2-4a02-9a22-044f2fbca6fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:53.280454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc767b6-cfc2-4a02-9a22-044f2fbca6fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:53.281818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc767b6-cfc2-4a02-9a22-044f2fbca6fb None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:53.289811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 613/1236] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Jul 25 12:29:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:53.295738 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4e46f-c46f-443e-ae07-e295c7f9b1f5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:53.387365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:53.405975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4e46f-c46f-443e-ae07-e295c7f9b1f5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:53.445082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc47d5-fc75-4bc2-8768-b84bd3771862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:53.445883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc47d5-fc75-4bc2-8768-b84bd3771862 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:53.446225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc47d5-fc75-4bc2-8768-b84bd3771862 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:53.446542 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc47d5-fc75-4bc2-8768-b84bd3771862 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:53.459257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc47d5-fc75-4bc2-8768-b84bd3771862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:53.460908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc47d5-fc75-4bc2-8768-b84bd3771862 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-6fbc47d5-fc75-4bc2-8768-b84bd3771862 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:29:53.488582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:53.490322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:53.491494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:29:53.491494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:29:53.491494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:53.491494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:29:53.491760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 624/1237] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:53.497881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc767b6-cfc2-4a02-9a22-044f2fbca6fb None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:53.612740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc767b6-cfc2-4a02-9a22-044f2fbca6fb None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:53.622924 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:53.653114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820aee50-b04d-4209-9d64-225640a1c34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:53.653707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aee50-b04d-4209-9d64-225640a1c34c None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:53.654018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aee50-b04d-4209-9d64-225640a1c34c None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:53.654296 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aee50-b04d-4209-9d64-225640a1c34c None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:53.655030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820aee50-b04d-4209-9d64-225640a1c34c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:53.655342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820aee50-b04d-4209-9d64-225640a1c34c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:53.656706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820aee50-b04d-4209-9d64-225640a1c34c None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:53.669493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 614/1238] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:53.677262 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc47d5-fc75-4bc2-8768-b84bd3771862 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:53.803489 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc47d5-fc75-4bc2-8768-b84bd3771862 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:53.846895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d91cf-3685-493e-86ff-6556c630e3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:53.847691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d91cf-3685-493e-86ff-6556c630e3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:53.847972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d91cf-3685-493e-86ff-6556c630e3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:53.848234 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d91cf-3685-493e-86ff-6556c630e3b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:53.852307 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:53.858087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d91cf-3685-493e-86ff-6556c630e3b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:53.859797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d91cf-3685-493e-86ff-6556c630e3b6 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-bb3d91cf-3685-493e-86ff-6556c630e3b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:29:53.878373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:53.879441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:53.881225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:53.881225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:53.881225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:29:53.881225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:29:53.881225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:29:53.881225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:29:53.881225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:29:53.881225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:29:53.881225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:29:53.881225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:29:53.881225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 625/1239] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:53.887100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820aee50-b04d-4209-9d64-225640a1c34c None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:54.035456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820aee50-b04d-4209-9d64-225640a1c34c None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:54.082495 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb1d5a-e7e5-4ccf-8b3e-26d7dabfef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:54.083174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb1d5a-e7e5-4ccf-8b3e-26d7dabfef54 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:54.083463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb1d5a-e7e5-4ccf-8b3e-26d7dabfef54 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:54.083742 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb1d5a-e7e5-4ccf-8b3e-26d7dabfef54 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:54.084695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb1d5a-e7e5-4ccf-8b3e-26d7dabfef54 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:54.085027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb1d5a-e7e5-4ccf-8b3e-26d7dabfef54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:54.086586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb1d5a-e7e5-4ccf-8b3e-26d7dabfef54 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:54.107145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 615/1240] 10.222.0.60 () {56 vars in 1143 bytes} [Mon Jul 25 12:29:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:54.114043 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d91cf-3685-493e-86ff-6556c630e3b6 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:54.235080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d91cf-3685-493e-86ff-6556c630e3b6 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:54.272680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41461a24-46f7-49fa-bab1-bc88196ecb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:54.273361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41461a24-46f7-49fa-bab1-bc88196ecb2d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:54.273641 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41461a24-46f7-49fa-bab1-bc88196ecb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:54.273994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41461a24-46f7-49fa-bab1-bc88196ecb2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:54.340466 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:54.350675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41461a24-46f7-49fa-bab1-bc88196ecb2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:54.352446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41461a24-46f7-49fa-bab1-bc88196ecb2d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:54.464413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 626/1241] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:29:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:29:54.470322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb1d5a-e7e5-4ccf-8b3e-26d7dabfef54 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:54.501791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:54.586978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb1d5a-e7e5-4ccf-8b3e-26d7dabfef54 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:54.631388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8d9c4-9e02-4aac-8ebe-3a0ce767bc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:54.632275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8d9c4-9e02-4aac-8ebe-3a0ce767bc50 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:54.632586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8d9c4-9e02-4aac-8ebe-3a0ce767bc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:54.632908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8d9c4-9e02-4aac-8ebe-3a0ce767bc50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:54.644144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8d9c4-9e02-4aac-8ebe-3a0ce767bc50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:54.645579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8d9c4-9e02-4aac-8ebe-3a0ce767bc50 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-5ab8d9c4-9e02-4aac-8ebe-3a0ce767bc50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:29:54.665558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:54.666761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:54.667676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:29:54.667676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:29:54.667676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:54.667676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:29:54.667676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 616/1242] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:54.672677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41461a24-46f7-49fa-bab1-bc88196ecb2d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:54.785552 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:54.792094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41461a24-46f7-49fa-bab1-bc88196ecb2d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:54.827739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa40f62-6441-4658-9c28-f2a3a0f8dd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:54.828479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40f62-6441-4658-9c28-f2a3a0f8dd56 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:54.828762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40f62-6441-4658-9c28-f2a3a0f8dd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:54.829068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40f62-6441-4658-9c28-f2a3a0f8dd56 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:54.829928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa40f62-6441-4658-9c28-f2a3a0f8dd56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:54.830434 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa40f62-6441-4658-9c28-f2a3a0f8dd56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:54.832420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa40f62-6441-4658-9c28-f2a3a0f8dd56 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:54.847627 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 627/1243] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:54.853218 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8d9c4-9e02-4aac-8ebe-3a0ce767bc50 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:54.944053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:54.971125 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8d9c4-9e02-4aac-8ebe-3a0ce767bc50 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:55.017874 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59330dc-203f-400d-b36c-874655d07bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:55.018781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59330dc-203f-400d-b36c-874655d07bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:55.019062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59330dc-203f-400d-b36c-874655d07bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:55.019335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59330dc-203f-400d-b36c-874655d07bb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:55.028954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59330dc-203f-400d-b36c-874655d07bb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:55.030542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59330dc-203f-400d-b36c-874655d07bb6 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-f59330dc-203f-400d-b36c-874655d07bb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:29:55.044587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:55.045711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:55.046920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:29:55.046920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:29:55.046920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:29:55.046920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:29:55.046920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:29:55.046920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:29:55.046920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:29:55.046920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:29:55.046920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 617/1244] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:29:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:55.052386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa40f62-6441-4658-9c28-f2a3a0f8dd56 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:55.166713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa40f62-6441-4658-9c28-f2a3a0f8dd56 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:55.206611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a6867-412a-4790-a594-0ead598bba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:55.207316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a6867-412a-4790-a594-0ead598bba34 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:55.207626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a6867-412a-4790-a594-0ead598bba34 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:55.207973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a6867-412a-4790-a594-0ead598bba34 None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:55.208763 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a6867-412a-4790-a594-0ead598bba34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:55.209080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a6867-412a-4790-a594-0ead598bba34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:55.210726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a6867-412a-4790-a594-0ead598bba34 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:55.223644 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 628/1245] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:29:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:55.229810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59330dc-203f-400d-b36c-874655d07bb6 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:55.343131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59330dc-203f-400d-b36c-874655d07bb6 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:55.380479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba2c1a-9ec3-47bc-a9dc-c4017634d4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:55.381147 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba2c1a-9ec3-47bc-a9dc-c4017634d4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:55.381433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba2c1a-9ec3-47bc-a9dc-c4017634d4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:55.381710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba2c1a-9ec3-47bc-a9dc-c4017634d4d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:55.382582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba2c1a-9ec3-47bc-a9dc-c4017634d4d5 None admin] RBAC: Adding query filter params (role.id=d9c5c1adb796486d83edcd6e728a5de9, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=81037f61e01e4a93982503720f7404e3) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:55.382892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba2c1a-9ec3-47bc-a9dc-c4017634d4d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:55.384333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba2c1a-9ec3-47bc-a9dc-c4017634d4d5 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:55.390921 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 618/1246] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:29:55 2022] GET /identity/v3/role_assignments?user.id=81037f61e01e4a93982503720f7404e3&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=d9c5c1adb796486d83edcd6e728a5de9 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:57.550122 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03345085-c6d4-4940-a9a0-0e0ae87a5a46 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:57.550680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03345085-c6d4-4940-a9a0-0e0ae87a5a46 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:57.550864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03345085-c6d4-4940-a9a0-0e0ae87a5a46 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:57.552469 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 629/1247] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:29:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:29:57.558552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed749e63-74a7-4730-8ada-faa710229f8f None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:57.559037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed749e63-74a7-4730-8ada-faa710229f8f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:57.559126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed749e63-74a7-4730-8ada-faa710229f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:57.565729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:57.579443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-ed749e63-74a7-4730-8ada-faa710229f8f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:29:57.580655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ed749e63-74a7-4730-8ada-faa710229f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:57.660803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 619/1248] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:29:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:29:57.683753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03345085-c6d4-4940-a9a0-0e0ae87a5a46 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:57.703937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:57.795547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03345085-c6d4-4940-a9a0-0e0ae87a5a46 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:57.834612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba575b39-f5a7-4ee7-b256-0453edf62496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:57.835335 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba575b39-f5a7-4ee7-b256-0453edf62496 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:57.835624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba575b39-f5a7-4ee7-b256-0453edf62496 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:57.835900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba575b39-f5a7-4ee7-b256-0453edf62496 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:57.847421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba575b39-f5a7-4ee7-b256-0453edf62496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-ba575b39-f5a7-4ee7-b256-0453edf62496 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:29:57.855982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:57.857332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:29:57.858592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 630/1249] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:29:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:57.863566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed749e63-74a7-4730-8ada-faa710229f8f None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:57.981690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed749e63-74a7-4730-8ada-faa710229f8f None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:58.018299 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:58.024730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751b15c-00c5-4449-9a81-8e453bf3a964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:58.025398 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751b15c-00c5-4449-9a81-8e453bf3a964 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:58.025678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751b15c-00c5-4449-9a81-8e453bf3a964 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:58.025974 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751b15c-00c5-4449-9a81-8e453bf3a964 None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:58.026729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751b15c-00c5-4449-9a81-8e453bf3a964 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:58.027028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751b15c-00c5-4449-9a81-8e453bf3a964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:58.028399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751b15c-00c5-4449-9a81-8e453bf3a964 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:58.037172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 620/1250] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:29:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:58.043414 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba575b39-f5a7-4ee7-b256-0453edf62496 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:58.126506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:58.152075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba575b39-f5a7-4ee7-b256-0453edf62496 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:58.192292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf565ba-56ba-4d68-b3e8-6bd57354f2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:58.193189 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf565ba-56ba-4d68-b3e8-6bd57354f2bf None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:58.193566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf565ba-56ba-4d68-b3e8-6bd57354f2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:58.193687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf565ba-56ba-4d68-b3e8-6bd57354f2bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:58.204737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf565ba-56ba-4d68-b3e8-6bd57354f2bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:58.206237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf565ba-56ba-4d68-b3e8-6bd57354f2bf None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-acf565ba-56ba-4d68-b3e8-6bd57354f2bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:29:58.227316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:58.228254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:58.229157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:29:58.229157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:29:58.229157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:58.229157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:29:58.229157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 631/1251] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:58.234978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751b15c-00c5-4449-9a81-8e453bf3a964 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:58.342875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751b15c-00c5-4449-9a81-8e453bf3a964 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:58.380453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7504920-8194-42db-bca3-42fe60ae939d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:58.381037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7504920-8194-42db-bca3-42fe60ae939d None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:58.381314 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7504920-8194-42db-bca3-42fe60ae939d None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:58.381590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7504920-8194-42db-bca3-42fe60ae939d None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:58.382391 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7504920-8194-42db-bca3-42fe60ae939d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:58.382680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7504920-8194-42db-bca3-42fe60ae939d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:58.384087 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7504920-8194-42db-bca3-42fe60ae939d None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:58.400413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 621/1252] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:58.406529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf565ba-56ba-4d68-b3e8-6bd57354f2bf None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:58.521834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf565ba-56ba-4d68-b3e8-6bd57354f2bf None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:58.549747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:58.558260 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef6b64-629e-4084-b437-e0d25e3d10a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:58.559822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef6b64-629e-4084-b437-e0d25e3d10a4 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:58.560088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef6b64-629e-4084-b437-e0d25e3d10a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:58.560406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef6b64-629e-4084-b437-e0d25e3d10a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:58.568736 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef6b64-629e-4084-b437-e0d25e3d10a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:58.570293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef6b64-629e-4084-b437-e0d25e3d10a4 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-abef6b64-629e-4084-b437-e0d25e3d10a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:29:58.586031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:58.587212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:58.588191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:58.588191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:58.588191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:29:58.588191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:29:58.588191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:29:58.588191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:29:58.588191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:29:58.588191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:29:58.588191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:29:58.588191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:29:58.588191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 632/1253] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:58.593460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7504920-8194-42db-bca3-42fe60ae939d None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:58.651324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:58.698139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7504920-8194-42db-bca3-42fe60ae939d None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:58.735005 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3050f13-cc64-4c15-8289-24d0530767de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:58.735754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3050f13-cc64-4c15-8289-24d0530767de None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:58.736060 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3050f13-cc64-4c15-8289-24d0530767de None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:58.736345 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3050f13-cc64-4c15-8289-24d0530767de None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:58.737299 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3050f13-cc64-4c15-8289-24d0530767de None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:58.737555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3050f13-cc64-4c15-8289-24d0530767de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:58.739614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3050f13-cc64-4c15-8289-24d0530767de None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:58.758545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 622/1254] 10.222.0.60 () {56 vars in 1143 bytes} [Mon Jul 25 12:29:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:29:58.764562 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef6b64-629e-4084-b437-e0d25e3d10a4 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:58.879136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef6b64-629e-4084-b437-e0d25e3d10a4 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:58.915828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5dd29-09b4-4890-96b1-a9db483d3d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:58.916493 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5dd29-09b4-4890-96b1-a9db483d3d8d None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:58.916784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5dd29-09b4-4890-96b1-a9db483d3d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:58.917156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5dd29-09b4-4890-96b1-a9db483d3d8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:58.984523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:58.998747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5dd29-09b4-4890-96b1-a9db483d3d8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:59.000199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5dd29-09b4-4890-96b1-a9db483d3d8d None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:59.105233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 633/1255] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:29:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:29:59.111704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3050f13-cc64-4c15-8289-24d0530767de None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:59.223899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3050f13-cc64-4c15-8289-24d0530767de None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:59.239812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:59.263721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b15eef-b24a-49e7-9923-8bd6e740a694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:59.264557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b15eef-b24a-49e7-9923-8bd6e740a694 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:59.264832 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b15eef-b24a-49e7-9923-8bd6e740a694 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:59.265101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b15eef-b24a-49e7-9923-8bd6e740a694 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:59.276540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b15eef-b24a-49e7-9923-8bd6e740a694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:59.278222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b15eef-b24a-49e7-9923-8bd6e740a694 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-04b15eef-b24a-49e7-9923-8bd6e740a694 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:29:59.300913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:59.302537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:59.303496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:29:59.303496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:29:59.303496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:29:59.303496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:29:59.303496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 623/1256] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:59.308617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5dd29-09b4-4890-96b1-a9db483d3d8d None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:59.400862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:59.413597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5dd29-09b4-4890-96b1-a9db483d3d8d None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:59.449719 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a77a5-1ef2-4d15-aa86-b2dfe7e2e190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:59.450398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a77a5-1ef2-4d15-aa86-b2dfe7e2e190 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:59.450701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a77a5-1ef2-4d15-aa86-b2dfe7e2e190 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:59.451003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a77a5-1ef2-4d15-aa86-b2dfe7e2e190 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:59.451818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a77a5-1ef2-4d15-aa86-b2dfe7e2e190 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:59.452127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a77a5-1ef2-4d15-aa86-b2dfe7e2e190 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:59.453615 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a77a5-1ef2-4d15-aa86-b2dfe7e2e190 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:59.467215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 634/1257] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:59.473406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b15eef-b24a-49e7-9923-8bd6e740a694 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:59.582130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b15eef-b24a-49e7-9923-8bd6e740a694 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:59.621954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f27367-3659-4eb8-b81b-e6d8297dc9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:59.622748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f27367-3659-4eb8-b81b-e6d8297dc9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:59.623074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f27367-3659-4eb8-b81b-e6d8297dc9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:59.623394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f27367-3659-4eb8-b81b-e6d8297dc9e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:59.631039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f27367-3659-4eb8-b81b-e6d8297dc9e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:59.632523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f27367-3659-4eb8-b81b-e6d8297dc9e4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-c4f27367-3659-4eb8-b81b-e6d8297dc9e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:29:59.646718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:29:59.647642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:29:59.648555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:29:59.648555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:29:59.648555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:29:59.648555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:29:59.648555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:29:59.648555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:29:59.648555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:29:59.648555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:29:59.648555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 624/1258] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:29:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:29:59.653955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a77a5-1ef2-4d15-aa86-b2dfe7e2e190 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:59.766183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a77a5-1ef2-4d15-aa86-b2dfe7e2e190 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:59.805179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a811d-bc48-4e2f-8234-fcabd3b2fdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:59.805872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a811d-bc48-4e2f-8234-fcabd3b2fdcc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:59.806186 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a811d-bc48-4e2f-8234-fcabd3b2fdcc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:59.806461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a811d-bc48-4e2f-8234-fcabd3b2fdcc None admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:59.807363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a811d-bc48-4e2f-8234-fcabd3b2fdcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:59.807709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a811d-bc48-4e2f-8234-fcabd3b2fdcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:59.809505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a811d-bc48-4e2f-8234-fcabd3b2fdcc None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:59.823708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 635/1259] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:29:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:29:59.830415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f27367-3659-4eb8-b81b-e6d8297dc9e4 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:59.859715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:59.948812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f27367-3659-4eb8-b81b-e6d8297dc9e4 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:59.988313 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c61a4-cee8-4fc0-a337-3f48f5457a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:59.989641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c61a4-cee8-4fc0-a337-3f48f5457a27 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:59.989995 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c61a4-cee8-4fc0-a337-3f48f5457a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:59.990280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c61a4-cee8-4fc0-a337-3f48f5457a27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:59.991108 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c61a4-cee8-4fc0-a337-3f48f5457a27 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=81037f61e01e4a93982503720f7404e3) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:29:59.991477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c61a4-cee8-4fc0-a337-3f48f5457a27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:59.993046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c61a4-cee8-4fc0-a337-3f48f5457a27 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:00.000088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 625/1260] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:29:59 2022] GET /identity/v3/role_assignments?user.id=81037f61e01e4a93982503720f7404e3&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:30:01.994337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad58f8-f37c-49b5-b11f-5e463e72ad9d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:01.994864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad58f8-f37c-49b5-b11f-5e463e72ad9d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:01.994864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad58f8-f37c-49b5-b11f-5e463e72ad9d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:01.996234 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 636/1261] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:30:02.001382 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0322e0-5099-4baf-a1fb-a8b723cb8bb6 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:02.001802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0322e0-5099-4baf-a1fb-a8b723cb8bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:02.001939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0322e0-5099-4baf-a1fb-a8b723cb8bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:02.019671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-db0322e0-5099-4baf-a1fb-a8b723cb8bb6 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:30:02.021359 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-db0322e0-5099-4baf-a1fb-a8b723cb8bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:30:02.084309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 626/1262] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:30:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:30:02.106772 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad58f8-f37c-49b5-b11f-5e463e72ad9d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:02.117380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:02.205376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad58f8-f37c-49b5-b11f-5e463e72ad9d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:02.238413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8e59d-a2b2-47ca-be28-ae4065827725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:02.239236 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8e59d-a2b2-47ca-be28-ae4065827725 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:02.239728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8e59d-a2b2-47ca-be28-ae4065827725 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:02.240006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8e59d-a2b2-47ca-be28-ae4065827725 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:02.249891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8e59d-a2b2-47ca-be28-ae4065827725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-92f8e59d-a2b2-47ca-be28-ae4065827725 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:30:02.257943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:02.259078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:30:02.260123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 637/1263] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:30:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:30:02.265077 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0322e0-5099-4baf-a1fb-a8b723cb8bb6 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:02.283269 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:02.360967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0322e0-5099-4baf-a1fb-a8b723cb8bb6 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:02.393487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62371437-df69-4f45-a96b-1f872effc0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:02.394233 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62371437-df69-4f45-a96b-1f872effc0eb None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:02.394542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62371437-df69-4f45-a96b-1f872effc0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:02.394819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62371437-df69-4f45-a96b-1f872effc0eb None admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:02.396158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62371437-df69-4f45-a96b-1f872effc0eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:02.396464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62371437-df69-4f45-a96b-1f872effc0eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:02.397814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62371437-df69-4f45-a96b-1f872effc0eb None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:02.405574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 627/1264] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:30:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:02.411168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8e59d-a2b2-47ca-be28-ae4065827725 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:02.476197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:02.509905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8e59d-a2b2-47ca-be28-ae4065827725 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:02.542720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db7c89-0021-43d9-ad5b-0c4ba491b62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:02.543498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db7c89-0021-43d9-ad5b-0c4ba491b62f None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:02.543776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db7c89-0021-43d9-ad5b-0c4ba491b62f None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:02.544048 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db7c89-0021-43d9-ad5b-0c4ba491b62f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:02.554809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db7c89-0021-43d9-ad5b-0c4ba491b62f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:02.556239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db7c89-0021-43d9-ad5b-0c4ba491b62f None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-b4db7c89-0021-43d9-ad5b-0c4ba491b62f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:30:02.573170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:02.574241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:02.575512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:30:02.575512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:30:02.575512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:30:02.575512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:30:02.575512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 638/1265] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:30:02.580000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62371437-df69-4f45-a96b-1f872effc0eb None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:02.657342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:02.675624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62371437-df69-4f45-a96b-1f872effc0eb None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:02.708520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3244b82-8d1b-4d46-ae13-74873ed0b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:02.709159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3244b82-8d1b-4d46-ae13-74873ed0b534 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:02.709438 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3244b82-8d1b-4d46-ae13-74873ed0b534 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:02.709709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3244b82-8d1b-4d46-ae13-74873ed0b534 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:02.710452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3244b82-8d1b-4d46-ae13-74873ed0b534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:02.710754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3244b82-8d1b-4d46-ae13-74873ed0b534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:02.712099 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3244b82-8d1b-4d46-ae13-74873ed0b534 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:02.723352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 628/1266] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:02.728884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db7c89-0021-43d9-ad5b-0c4ba491b62f None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:02.828783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db7c89-0021-43d9-ad5b-0c4ba491b62f None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:02.847580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:02.861399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5151a-d1ba-4fec-a5f7-6b823facfb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:02.862222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5151a-d1ba-4fec-a5f7-6b823facfb51 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:02.862499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5151a-d1ba-4fec-a5f7-6b823facfb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:02.862765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5151a-d1ba-4fec-a5f7-6b823facfb51 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:02.870333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5151a-d1ba-4fec-a5f7-6b823facfb51 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:02.871759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5151a-d1ba-4fec-a5f7-6b823facfb51 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-90a5151a-d1ba-4fec-a5f7-6b823facfb51 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:30:02.886120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:02.887233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:02.888223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:02.888223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:02.888223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:30:02.888223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:30:02.888223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:30:02.888223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:30:02.888223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:30:02.888223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:30:02.888223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:30:02.888223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:30:02.888223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 639/1267] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:30:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:30:02.893399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3244b82-8d1b-4d46-ae13-74873ed0b534 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:02.999410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3244b82-8d1b-4d46-ae13-74873ed0b534 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:03.033659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a34f6-dda2-4f6d-ad8e-8adc40415da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:03.034341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a34f6-dda2-4f6d-ad8e-8adc40415da5 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:03.034618 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a34f6-dda2-4f6d-ad8e-8adc40415da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:03.034888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a34f6-dda2-4f6d-ad8e-8adc40415da5 None admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:03.036245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a34f6-dda2-4f6d-ad8e-8adc40415da5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:03.036548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a34f6-dda2-4f6d-ad8e-8adc40415da5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:03.037974 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a34f6-dda2-4f6d-ad8e-8adc40415da5 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:03.046176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:03.055353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 629/1268] 10.222.0.60 () {56 vars in 1143 bytes} [Mon Jul 25 12:30:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:03.061078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5151a-d1ba-4fec-a5f7-6b823facfb51 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:03.170146 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5151a-d1ba-4fec-a5f7-6b823facfb51 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:03.203047 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b220291-4ee7-4b0f-b102-6f70f01b7ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:03.203838 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b220291-4ee7-4b0f-b102-6f70f01b7ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:03.204123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b220291-4ee7-4b0f-b102-6f70f01b7ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:03.204396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b220291-4ee7-4b0f-b102-6f70f01b7ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:03.214651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b220291-4ee7-4b0f-b102-6f70f01b7ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:03.216088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b220291-4ee7-4b0f-b102-6f70f01b7ef5 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-3b220291-4ee7-4b0f-b102-6f70f01b7ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:30:03.233479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:03.234499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:03.235581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:30:03.235581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:30:03.235581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:30:03.235581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:30:03.235581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 640/1269] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:30:03.241303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a34f6-dda2-4f6d-ad8e-8adc40415da5 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:03.335870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a34f6-dda2-4f6d-ad8e-8adc40415da5 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:03.368702 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b53218-eb99-49a9-b449-8362be82cdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:03.369348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53218-eb99-49a9-b449-8362be82cdbf None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:03.369627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53218-eb99-49a9-b449-8362be82cdbf None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:03.369898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53218-eb99-49a9-b449-8362be82cdbf None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:03.370631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b53218-eb99-49a9-b449-8362be82cdbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:03.370933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b53218-eb99-49a9-b449-8362be82cdbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:03.372276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b53218-eb99-49a9-b449-8362be82cdbf None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:03.382989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 630/1270] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:03.388505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b220291-4ee7-4b0f-b102-6f70f01b7ef5 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:03.419840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:03.485674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b220291-4ee7-4b0f-b102-6f70f01b7ef5 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:03.517166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105e56c-6e76-43f6-bffb-6c51eec48d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:03.517961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105e56c-6e76-43f6-bffb-6c51eec48d82 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:03.518247 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105e56c-6e76-43f6-bffb-6c51eec48d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:03.518524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105e56c-6e76-43f6-bffb-6c51eec48d82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:03.525459 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105e56c-6e76-43f6-bffb-6c51eec48d82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:03.526914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105e56c-6e76-43f6-bffb-6c51eec48d82 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-5105e56c-6e76-43f6-bffb-6c51eec48d82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:30:03.541311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:03.542397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:03.543673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:30:03.543673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:30:03.543673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:30:03.543673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:30:03.543673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:30:03.543673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:30:03.543673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:30:03.543673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:30:03.543673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 641/1271] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:30:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:30:03.548209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b53218-eb99-49a9-b449-8362be82cdbf None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:03.604946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:03.646929 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b53218-eb99-49a9-b449-8362be82cdbf None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:03.678694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52efa3-29f7-4d96-a8a9-6b4c56293b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:03.679341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52efa3-29f7-4d96-a8a9-6b4c56293b36 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:03.679622 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52efa3-29f7-4d96-a8a9-6b4c56293b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:03.679896 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52efa3-29f7-4d96-a8a9-6b4c56293b36 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:03.680675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52efa3-29f7-4d96-a8a9-6b4c56293b36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:03.680981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52efa3-29f7-4d96-a8a9-6b4c56293b36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:03.682433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52efa3-29f7-4d96-a8a9-6b4c56293b36 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:03.694404 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 631/1272] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:30:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:03.700629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105e56c-6e76-43f6-bffb-6c51eec48d82 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:03.787953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:03.796553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105e56c-6e76-43f6-bffb-6c51eec48d82 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:03.828750 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b818ba-c2b6-4c2c-a820-abb76dfa453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:03.829612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b818ba-c2b6-4c2c-a820-abb76dfa453e None admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:03.829891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b818ba-c2b6-4c2c-a820-abb76dfa453e None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:03.830207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b818ba-c2b6-4c2c-a820-abb76dfa453e None admin] PATH_INFO: `/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/81037f61e01e4a93982503720f7404e3/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:03.832081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b818ba-c2b6-4c2c-a820-abb76dfa453e None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff5a41f8ca6438295c512792d2022e4, user_id=81037f61e01e4a93982503720f7404e3, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:03.834133 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b818ba-c2b6-4c2c-a820-abb76dfa453e None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:03.845998 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 642/1273] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Jul 25 12:30:03 2022] PUT /identity/v3/projects/9ff5a41f8ca6438295c512792d2022e4/users/81037f61e01e4a93982503720f7404e3/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:30:05.805797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0f915-7841-47bd-ba26-04b7ea455a75 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:05.806359 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0f915-7841-47bd-ba26-04b7ea455a75 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:05.806359 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0f915-7841-47bd-ba26-04b7ea455a75 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:05.807740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 632/1274] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:30:05.812761 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6adad-7b31-4f81-848e-9a4c7b0c5d8d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:05.813162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6adad-7b31-4f81-848e-9a4c7b0c5d8d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:05.813319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6adad-7b31-4f81-848e-9a4c7b0c5d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:05.831583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a4e6adad-7b31-4f81-848e-9a4c7b0c5d8d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:30:05.832766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a4e6adad-7b31-4f81-848e-9a4c7b0c5d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:30:05.897535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 643/1275] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:30:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:30:05.920670 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0f915-7841-47bd-ba26-04b7ea455a75 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:06.036738 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0f915-7841-47bd-ba26-04b7ea455a75 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:06.053538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:06.069823 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebeac8c-4ddf-47e4-a523-e48a37b2282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:06.070570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebeac8c-4ddf-47e4-a523-e48a37b2282e None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:06.070847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebeac8c-4ddf-47e4-a523-e48a37b2282e None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:06.071116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebeac8c-4ddf-47e4-a523-e48a37b2282e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:06.080757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebeac8c-4ddf-47e4-a523-e48a37b2282e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-6ebeac8c-4ddf-47e4-a523-e48a37b2282e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:30:06.088410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:06.089479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:30:06.090620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 633/1276] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Jul 25 12:30:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:30:06.095663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6adad-7b31-4f81-848e-9a4c7b0c5d8d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:06.173630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:06.192236 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6adad-7b31-4f81-848e-9a4c7b0c5d8d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:06.224942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e6179-200e-4cf0-a0fe-b39adcd44a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:06.225596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e6179-200e-4cf0-a0fe-b39adcd44a66 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:06.225877 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e6179-200e-4cf0-a0fe-b39adcd44a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:06.226187 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e6179-200e-4cf0-a0fe-b39adcd44a66 None admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:06.226924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e6179-200e-4cf0-a0fe-b39adcd44a66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:06.227228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e6179-200e-4cf0-a0fe-b39adcd44a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:06.228582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e6179-200e-4cf0-a0fe-b39adcd44a66 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:06.236701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 644/1277] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Jul 25 12:30:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:30:06.242988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebeac8c-4ddf-47e4-a523-e48a37b2282e None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:06.339016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebeac8c-4ddf-47e4-a523-e48a37b2282e None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:06.371525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fa1d0-f0ae-4967-a110-b34c4301b9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:06.372284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fa1d0-f0ae-4967-a110-b34c4301b9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:06.372567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fa1d0-f0ae-4967-a110-b34c4301b9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:06.372843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fa1d0-f0ae-4967-a110-b34c4301b9c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:06.383753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fa1d0-f0ae-4967-a110-b34c4301b9c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:06.385185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fa1d0-f0ae-4967-a110-b34c4301b9c4 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-e08fa1d0-f0ae-4967-a110-b34c4301b9c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:30:06.402046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:06.403055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:06.404146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:30:06.404146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:30:06.404146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:30:06.404146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:30:06.404146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 634/1278] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:30:06.408825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e6179-200e-4cf0-a0fe-b39adcd44a66 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:06.504875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e6179-200e-4cf0-a0fe-b39adcd44a66 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:06.537331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35106d2b-ec1a-47c0-9dd5-fd27504836b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:06.538001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35106d2b-ec1a-47c0-9dd5-fd27504836b3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:06.538287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35106d2b-ec1a-47c0-9dd5-fd27504836b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:06.538564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35106d2b-ec1a-47c0-9dd5-fd27504836b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:06.539272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35106d2b-ec1a-47c0-9dd5-fd27504836b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:06.539577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35106d2b-ec1a-47c0-9dd5-fd27504836b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:06.540930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35106d2b-ec1a-47c0-9dd5-fd27504836b3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:06.552218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 645/1279] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:06.557770 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fa1d0-f0ae-4967-a110-b34c4301b9c4 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:06.587806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:06.655349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fa1d0-f0ae-4967-a110-b34c4301b9c4 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:06.686785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026eb11c-433a-4195-864a-ce189262246b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:06.687563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026eb11c-433a-4195-864a-ce189262246b None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:06.687846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026eb11c-433a-4195-864a-ce189262246b None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:06.688121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026eb11c-433a-4195-864a-ce189262246b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:06.695950 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026eb11c-433a-4195-864a-ce189262246b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:06.697372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026eb11c-433a-4195-864a-ce189262246b None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-026eb11c-433a-4195-864a-ce189262246b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:30:06.711773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:06.712856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:06.714022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:06.714022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:06.714022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:30:06.714022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:30:06.714022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:30:06.714022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:30:06.714022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:30:06.714022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:30:06.714022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:30:06.714022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:30:06.714022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 635/1280] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:30:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:30:06.718712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35106d2b-ec1a-47c0-9dd5-fd27504836b3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:06.722376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:06.816952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35106d2b-ec1a-47c0-9dd5-fd27504836b3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:06.849595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1d785-b8fd-4044-ad18-b1d6e4d36acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:06.850272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1d785-b8fd-4044-ad18-b1d6e4d36acc None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:06.850555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1d785-b8fd-4044-ad18-b1d6e4d36acc None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:06.850831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1d785-b8fd-4044-ad18-b1d6e4d36acc None admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:06.851657 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1d785-b8fd-4044-ad18-b1d6e4d36acc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:06.851965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1d785-b8fd-4044-ad18-b1d6e4d36acc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:06.853381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1d785-b8fd-4044-ad18-b1d6e4d36acc None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:06.869987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 646/1281] 10.222.0.60 () {56 vars in 1143 bytes} [Mon Jul 25 12:30:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:06.875644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026eb11c-433a-4195-864a-ce189262246b None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:06.967109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:06.976665 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026eb11c-433a-4195-864a-ce189262246b None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:07.009410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030d90e-c01a-49e3-82e6-70f6ff15bee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:07.010096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030d90e-c01a-49e3-82e6-70f6ff15bee0 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:07.010378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030d90e-c01a-49e3-82e6-70f6ff15bee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:07.010651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030d90e-c01a-49e3-82e6-70f6ff15bee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:07.074883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030d90e-c01a-49e3-82e6-70f6ff15bee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:07.076310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030d90e-c01a-49e3-82e6-70f6ff15bee0 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:07.164835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:07.175385 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 636/1282] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Jul 25 12:30:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:30:07.181537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1d785-b8fd-4044-ad18-b1d6e4d36acc None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:07.241500 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:07.278253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1d785-b8fd-4044-ad18-b1d6e4d36acc None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:07.310638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d2410-d0f0-428b-bb22-bbbdd35284f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:07.311421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d2410-d0f0-428b-bb22-bbbdd35284f3 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:07.311700 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d2410-d0f0-428b-bb22-bbbdd35284f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:07.311969 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d2410-d0f0-428b-bb22-bbbdd35284f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:07.322199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d2410-d0f0-428b-bb22-bbbdd35284f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:07.323643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d2410-d0f0-428b-bb22-bbbdd35284f3 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-3b7d2410-d0f0-428b-bb22-bbbdd35284f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:30:07.340419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:07.341421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:07.342398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:30:07.342398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:30:07.342398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:30:07.342398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:30:07.342398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 647/1283] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:30:07.347231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030d90e-c01a-49e3-82e6-70f6ff15bee0 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:07.441017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030d90e-c01a-49e3-82e6-70f6ff15bee0 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:07.472608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29f4fa-949b-4846-9df6-06af7dc9e5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:07.473244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29f4fa-949b-4846-9df6-06af7dc9e5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:07.473522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29f4fa-949b-4846-9df6-06af7dc9e5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:07.473796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29f4fa-949b-4846-9df6-06af7dc9e5d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:07.475035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29f4fa-949b-4846-9df6-06af7dc9e5d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:07.475337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29f4fa-949b-4846-9df6-06af7dc9e5d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:07.476700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29f4fa-949b-4846-9df6-06af7dc9e5d1 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:07.487652 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 637/1284] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Jul 25 12:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:07.493715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d2410-d0f0-428b-bb22-bbbdd35284f3 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:07.587646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d2410-d0f0-428b-bb22-bbbdd35284f3 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:07.601341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:07.622258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b3920-9574-48ca-81ca-a422fd0a0d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:07.623022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b3920-9574-48ca-81ca-a422fd0a0d89 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:07.623301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b3920-9574-48ca-81ca-a422fd0a0d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:07.623573 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b3920-9574-48ca-81ca-a422fd0a0d89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:07.629954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b3920-9574-48ca-81ca-a422fd0a0d89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:07.631399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b3920-9574-48ca-81ca-a422fd0a0d89 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application [None req-017b3920-9574-48ca-81ca-a422fd0a0d89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:30:07.645997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:07.647046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:07.648027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:30:07.648027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:30:07.648027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:30:07.648027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:30:07.648027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:30:07.648027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:30:07.648027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:30:07.648027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: ERROR keystone.server.flask.application Jul 25 12:30:07.648027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 648/1285] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Jul 25 12:30:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:30:07.652901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29f4fa-949b-4846-9df6-06af7dc9e5d1 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:07.701258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:07.749901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29f4fa-949b-4846-9df6-06af7dc9e5d1 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:07.781614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f10f30-fd87-4847-9bec-7381d04ea536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:07.782276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10f30-fd87-4847-9bec-7381d04ea536 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:07.782555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10f30-fd87-4847-9bec-7381d04ea536 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:07.782827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10f30-fd87-4847-9bec-7381d04ea536 None admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:07.783597 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f10f30-fd87-4847-9bec-7381d04ea536 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:07.783899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f10f30-fd87-4847-9bec-7381d04ea536 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:07.785309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f10f30-fd87-4847-9bec-7381d04ea536 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:07.797239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 638/1286] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Jul 25 12:30:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:07.803246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b3920-9574-48ca-81ca-a422fd0a0d89 None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:07.898954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b3920-9574-48ca-81ca-a422fd0a0d89 None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:07.930869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469ff39-6e16-4bab-a3a0-65850e1a4712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:07.931492 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469ff39-6e16-4bab-a3a0-65850e1a4712 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:07.931770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469ff39-6e16-4bab-a3a0-65850e1a4712 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:07.932043 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469ff39-6e16-4bab-a3a0-65850e1a4712 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:07.932857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469ff39-6e16-4bab-a3a0-65850e1a4712 None admin] RBAC: Adding query filter params (role.id=dc72adb78cf8456d9f0a66875d92c83a, scope.project.id=9ff5a41f8ca6438295c512792d2022e4, user.id=81037f61e01e4a93982503720f7404e3) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:07.933161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469ff39-6e16-4bab-a3a0-65850e1a4712 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:07.934597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469ff39-6e16-4bab-a3a0-65850e1a4712 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:07.941054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 649/1287] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Jul 25 12:30:07 2022] GET /identity/v3/role_assignments?user.id=81037f61e01e4a93982503720f7404e3&scope.project.id=9ff5a41f8ca6438295c512792d2022e4&role.id=dc72adb78cf8456d9f0a66875d92c83a => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:09.930794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659614ea-0515-424a-a93a-d63499850f56 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:09.931318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659614ea-0515-424a-a93a-d63499850f56 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:09.931318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659614ea-0515-424a-a93a-d63499850f56 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:09.932703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 639/1288] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:30:09.938169 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf0311-be98-4ab1-b1ac-a21f38f81eb6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:09.938581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf0311-be98-4ab1-b1ac-a21f38f81eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:09.938898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf0311-be98-4ab1-b1ac-a21f38f81eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:09.956888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b2cf0311-be98-4ab1-b1ac-a21f38f81eb6 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:30:09.958091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b2cf0311-be98-4ab1-b1ac-a21f38f81eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:30:09.987317 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:10.023069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 650/1289] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:30:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:30:10.045024 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659614ea-0515-424a-a93a-d63499850f56 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:10.141219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659614ea-0515-424a-a93a-d63499850f56 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:10.151919 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:10.174928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fab76-8726-4756-aebe-b5666f2ed387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:10.175715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fab76-8726-4756-aebe-b5666f2ed387 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:10.175996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fab76-8726-4756-aebe-b5666f2ed387 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:10.176272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fab76-8726-4756-aebe-b5666f2ed387 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:10.176945 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fab76-8726-4756-aebe-b5666f2ed387 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:10.178351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fab76-8726-4756-aebe-b5666f2ed387 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-096fab76-8726-4756-aebe-b5666f2ed387 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:10.189987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:10.191028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:30:10.192000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 640/1290] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Jul 25 12:30:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:30:10.197327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf0311-be98-4ab1-b1ac-a21f38f81eb6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:10.292871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf0311-be98-4ab1-b1ac-a21f38f81eb6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:10.324753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fece222-a405-4b5f-8664-2948748f6e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:10.325398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fece222-a405-4b5f-8664-2948748f6e21 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:10.325683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fece222-a405-4b5f-8664-2948748f6e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:10.325985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fece222-a405-4b5f-8664-2948748f6e21 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:10.326692 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fece222-a405-4b5f-8664-2948748f6e21 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:10.326997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fece222-a405-4b5f-8664-2948748f6e21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:10.328362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fece222-a405-4b5f-8664-2948748f6e21 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:10.333066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 651/1291] 10.222.0.60 () {56 vars in 1116 bytes} [Mon Jul 25 12:30:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:10.339105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fab76-8726-4756-aebe-b5666f2ed387 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:10.434386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fab76-8726-4756-aebe-b5666f2ed387 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:10.466895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270beef7-26f4-452b-aaad-0109f0d1c5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:10.467481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270beef7-26f4-452b-aaad-0109f0d1c5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:10.467762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270beef7-26f4-452b-aaad-0109f0d1c5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:10.468037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270beef7-26f4-452b-aaad-0109f0d1c5d1 None admin] PATH_INFO: `/v3/services` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:10.468737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270beef7-26f4-452b-aaad-0109f0d1c5d1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:10.469044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270beef7-26f4-452b-aaad-0109f0d1c5d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:10.470419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270beef7-26f4-452b-aaad-0109f0d1c5d1 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:10.474735 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 641/1292] 10.222.0.60 () {56 vars in 1116 bytes} [Mon Jul 25 12:30:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:12.460688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfcd26-366a-4463-9dcf-fa6269fd0897 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:12.461206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfcd26-366a-4463-9dcf-fa6269fd0897 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:12.461206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfcd26-366a-4463-9dcf-fa6269fd0897 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:12.462545 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 652/1293] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:30:12.467695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c7487-efe6-4665-9da3-3323f9033761 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:12.468056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c7487-efe6-4665-9da3-3323f9033761 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:12.468248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c7487-efe6-4665-9da3-3323f9033761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:12.486627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-131c7487-efe6-4665-9da3-3323f9033761 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:30:12.487788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-131c7487-efe6-4665-9da3-3323f9033761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:30:12.515722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:12.555481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 642/1294] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:30:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:30:12.577363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfcd26-366a-4463-9dcf-fa6269fd0897 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:12.608481 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:12.676585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfcd26-366a-4463-9dcf-fa6269fd0897 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:12.710444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40107787-bc81-49a5-94b5-7b3dacb84ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:12.711148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40107787-bc81-49a5-94b5-7b3dacb84ad0 None admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:12.711375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40107787-bc81-49a5-94b5-7b3dacb84ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:12.711646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40107787-bc81-49a5-94b5-7b3dacb84ad0 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:12.712365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40107787-bc81-49a5-94b5-7b3dacb84ad0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:12.713722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40107787-bc81-49a5-94b5-7b3dacb84ad0 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:12.727379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 653/1295] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Jul 25 12:30:12 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:30:14.692074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ef37c-2e3e-4275-8609-13888f500d43 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:14.693150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ef37c-2e3e-4275-8609-13888f500d43 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:14.693255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ef37c-2e3e-4275-8609-13888f500d43 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:14.694662 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 643/1296] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:30:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:30:14.699663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93d206-9789-4dce-8dc1-5a7d19f5265e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:14.700097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93d206-9789-4dce-8dc1-5a7d19f5265e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:14.700218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93d206-9789-4dce-8dc1-5a7d19f5265e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:14.717775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4a93d206-9789-4dce-8dc1-5a7d19f5265e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:30:14.718988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4a93d206-9789-4dce-8dc1-5a7d19f5265e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:30:14.788557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 654/1297] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:30:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:30:14.810394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ef37c-2e3e-4275-8609-13888f500d43 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:14.908822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ef37c-2e3e-4275-8609-13888f500d43 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:14.942425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e5bbc-1e45-4ce7-9361-b1769cad91c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:14.943193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e5bbc-1e45-4ce7-9361-b1769cad91c6 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:14.943474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e5bbc-1e45-4ce7-9361-b1769cad91c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:14.943754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e5bbc-1e45-4ce7-9361-b1769cad91c6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:14.944710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e5bbc-1e45-4ce7-9361-b1769cad91c6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:14.946072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e5bbc-1e45-4ce7-9361-b1769cad91c6 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-2e9e5bbc-1e45-4ce7-9361-b1769cad91c6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:14.959164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:14.960530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:30:14.961845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 644/1298] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Jul 25 12:30:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:30:14.967278 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93d206-9789-4dce-8dc1-5a7d19f5265e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:14.989322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:15.074530 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93d206-9789-4dce-8dc1-5a7d19f5265e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:15.108180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a570f-3f12-4086-a5e0-e8c4e1485afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:15.108832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a570f-3f12-4086-a5e0-e8c4e1485afa None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:15.109111 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a570f-3f12-4086-a5e0-e8c4e1485afa None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:15.109383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a570f-3f12-4086-a5e0-e8c4e1485afa None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:15.110101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a570f-3f12-4086-a5e0-e8c4e1485afa None admin] RBAC: Adding query filter params (name=placement) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:15.110406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a570f-3f12-4086-a5e0-e8c4e1485afa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:15.111756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a570f-3f12-4086-a5e0-e8c4e1485afa None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:15.117344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 655/1299] 10.222.0.60 () {56 vars in 1116 bytes} [Mon Jul 25 12:30:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:15.123322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e5bbc-1e45-4ce7-9361-b1769cad91c6 None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:15.156736 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:15.224303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e5bbc-1e45-4ce7-9361-b1769cad91c6 None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:15.255917 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33730b0d-20bc-486c-a31b-cbdd80563ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:15.256560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33730b0d-20bc-486c-a31b-cbdd80563ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:15.256841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33730b0d-20bc-486c-a31b-cbdd80563ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:15.257114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33730b0d-20bc-486c-a31b-cbdd80563ae2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:15.257891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33730b0d-20bc-486c-a31b-cbdd80563ae2 None admin] RBAC: Adding query filter params (interface=public, service_id=f5a0825af95b4162bf7473e599971cdb, region_id=RegionOne) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:15.258229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33730b0d-20bc-486c-a31b-cbdd80563ae2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:15.259573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33730b0d-20bc-486c-a31b-cbdd80563ae2 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:15.264393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 645/1300] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Jul 25 12:30:15 2022] GET /identity/v3/endpoints?service_id=f5a0825af95b4162bf7473e599971cdb&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:15.270126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a570f-3f12-4086-a5e0-e8c4e1485afa None admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:15.349768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:15.367735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a570f-3f12-4086-a5e0-e8c4e1485afa None admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:15.400280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accbada-f33b-4992-90be-c07a7f2f8c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:15.400929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accbada-f33b-4992-90be-c07a7f2f8c01 None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:15.401217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accbada-f33b-4992-90be-c07a7f2f8c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:15.401495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accbada-f33b-4992-90be-c07a7f2f8c01 None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:15.402200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accbada-f33b-4992-90be-c07a7f2f8c01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:15.403574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accbada-f33b-4992-90be-c07a7f2f8c01 None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:15.410242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 656/1301] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Jul 25 12:30:15 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:30:17.382794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7ba1d-8665-4fdf-99e0-ea046c1c218c None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:17.383321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7ba1d-8665-4fdf-99e0-ea046c1c218c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:17.383321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7ba1d-8665-4fdf-99e0-ea046c1c218c None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:17.384706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 646/1302] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:30:17.389652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c3cfc-7c1a-4f67-8b7e-acbc23aa11cc None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:17.390046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c3cfc-7c1a-4f67-8b7e-acbc23aa11cc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:17.390237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c3cfc-7c1a-4f67-8b7e-acbc23aa11cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:17.407703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c08c3cfc-7c1a-4f67-8b7e-acbc23aa11cc None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:30:17.408867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c08c3cfc-7c1a-4f67-8b7e-acbc23aa11cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:30:17.472823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 657/1303] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:30:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:30:17.495223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7ba1d-8665-4fdf-99e0-ea046c1c218c None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:17.587292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:17.594087 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7ba1d-8665-4fdf-99e0-ea046c1c218c None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:17.627253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bdf34-1ba6-4696-bae9-f8e23e36ebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:17.628031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bdf34-1ba6-4696-bae9-f8e23e36ebfd None admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:17.628311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bdf34-1ba6-4696-bae9-f8e23e36ebfd None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:17.628583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bdf34-1ba6-4696-bae9-f8e23e36ebfd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:17.629228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bdf34-1ba6-4696-bae9-f8e23e36ebfd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:17.631163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bdf34-1ba6-4696-bae9-f8e23e36ebfd None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application [None req-251bdf34-1ba6-4696-bae9-f8e23e36ebfd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:30:17.642668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:30:17.643827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application with Lock( Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: ERROR keystone.server.flask.application Jul 25 12:30:17.644737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 647/1304] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Jul 25 12:30:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:30:17.649846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c3cfc-7c1a-4f67-8b7e-acbc23aa11cc None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:17.730985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:17.753767 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c3cfc-7c1a-4f67-8b7e-acbc23aa11cc None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:17.786499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7f511-731c-41f3-adc1-9fcf852ccdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:17.787153 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7f511-731c-41f3-adc1-9fcf852ccdfb None admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:17.787432 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7f511-731c-41f3-adc1-9fcf852ccdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:17.787704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7f511-731c-41f3-adc1-9fcf852ccdfb None admin] PATH_INFO: `/v3/services` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:17.788404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7f511-731c-41f3-adc1-9fcf852ccdfb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:30:17.788709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7f511-731c-41f3-adc1-9fcf852ccdfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:17.790092 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7f511-731c-41f3-adc1-9fcf852ccdfb None admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:17.795108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 658/1305] 10.222.0.60 () {56 vars in 1116 bytes} [Mon Jul 25 12:30:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:30:17.800832 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bdf34-1ba6-4696-bae9-f8e23e36ebfd None admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:17.896986 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bdf34-1ba6-4696-bae9-f8e23e36ebfd None admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:17.929336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8717d1-e289-47cc-b5fc-5b9c16478138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:17.929992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8717d1-e289-47cc-b5fc-5b9c16478138 None admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:17.930276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8717d1-e289-47cc-b5fc-5b9c16478138 None admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:17.930550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8717d1-e289-47cc-b5fc-5b9c16478138 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:17.931291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8717d1-e289-47cc-b5fc-5b9c16478138 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:17.932638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8717d1-e289-47cc-b5fc-5b9c16478138 None admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:17.950666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 648/1306] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:30:17 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:30:27.792084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=30 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:27.950601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:20.789175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c98d5-6e01-4c20-886b-9ab7053f896d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:20.789749 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c98d5-6e01-4c20-886b-9ab7053f896d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:20.789749 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c98d5-6e01-4c20-886b-9ab7053f896d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:20.794870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 659/1307] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:20.796519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056dab7d-d34f-4eaf-8d4d-a93048810783 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:20.797362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056dab7d-d34f-4eaf-8d4d-a93048810783 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:20.797806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056dab7d-d34f-4eaf-8d4d-a93048810783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:20.900445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-056dab7d-d34f-4eaf-8d4d-a93048810783 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:20.901937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-056dab7d-d34f-4eaf-8d4d-a93048810783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:21.059137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 649/1308] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:31:21.110782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab159b89-b072-4e8e-80c7-ce34c88b672b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:21.111270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab159b89-b072-4e8e-80c7-ce34c88b672b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:21.111630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab159b89-b072-4e8e-80c7-ce34c88b672b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:21.113509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 660/1309] 10.222.0.60 () {54 vars in 965 bytes} [Mon Jul 25 12:31:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:21.119696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83079f2-ff3b-44f7-a058-4d81659a75f5 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:21.119995 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83079f2-ff3b-44f7-a058-4d81659a75f5 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:21.120281 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83079f2-ff3b-44f7-a058-4d81659a75f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:21.223592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-a83079f2-ff3b-44f7-a058-4d81659a75f5 None None] MFA Rules not processed for user `375d321d6de2494794254e50151c02c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:21.225219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a83079f2-ff3b-44f7-a058-4d81659a75f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:21.250429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:21.310339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 650/1310] 10.222.0.60 () {58 vars in 1062 bytes} [Mon Jul 25 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:31:21.320250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab159b89-b072-4e8e-80c7-ce34c88b672b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:31:21.322110 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ab159b89-b072-4e8e-80c7-ce34c88b672b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:21.436025 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab159b89-b072-4e8e-80c7-ce34c88b672b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:31:21.512656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e86ed8-5142-47ce-ab61-4e4693455f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:31:21.513225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e86ed8-5142-47ce-ab61-4e4693455f42 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:21.513507 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e86ed8-5142-47ce-ab61-4e4693455f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:21.513776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e86ed8-5142-47ce-ab61-4e4693455f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:21.515439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c6e86ed8-5142-47ce-ab61-4e4693455f42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:21.576132 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:21.633801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e86ed8-5142-47ce-ab61-4e4693455f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:31:21.635344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e86ed8-5142-47ce-ab61-4e4693455f42 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:31:21.654565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 661/1311] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:31:24.073160 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1d342-5d4d-4463-9ace-854b0c12a08f None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:24.073713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1d342-5d4d-4463-9ace-854b0c12a08f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:24.073713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1d342-5d4d-4463-9ace-854b0c12a08f None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:24.075259 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 651/1312] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:24.080352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4df1c-9af1-493c-9e0f-d7f09f575a43 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:24.080763 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4df1c-9af1-493c-9e0f-d7f09f575a43 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:24.080914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4df1c-9af1-493c-9e0f-d7f09f575a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:24.154358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-60f4df1c-9af1-493c-9e0f-d7f09f575a43 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:24.155749 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-60f4df1c-9af1-493c-9e0f-d7f09f575a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:24.263195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 662/1313] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:31:24.301608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1d342-5d4d-4463-9ace-854b0c12a08f None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:31:24.303195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d8d1d342-5d4d-4463-9ace-854b0c12a08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:24.414420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1d342-5d4d-4463-9ace-854b0c12a08f None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:31:24.497432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:24.500503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0950f5-df8d-41de-b16b-65327a2e3944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:31:24.501193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0950f5-df8d-41de-b16b-65327a2e3944 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:24.501475 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0950f5-df8d-41de-b16b-65327a2e3944 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:24.501828 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0950f5-df8d-41de-b16b-65327a2e3944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:24.504279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8a0950f5-df8d-41de-b16b-65327a2e3944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:24.631607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0950f5-df8d-41de-b16b-65327a2e3944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:31:24.633168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0950f5-df8d-41de-b16b-65327a2e3944 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:31:24.653261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 652/1314] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:31:26.995625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596939c3-08c4-4f6e-8752-635df0808951 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:26.996178 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596939c3-08c4-4f6e-8752-635df0808951 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:26.996178 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596939c3-08c4-4f6e-8752-635df0808951 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:26.997555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 663/1315] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:27.003133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39650f7e-c5a8-497f-bcb6-85c8200d8dce None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:27.003554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39650f7e-c5a8-497f-bcb6-85c8200d8dce None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:27.003690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39650f7e-c5a8-497f-bcb6-85c8200d8dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:27.068184 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-39650f7e-c5a8-497f-bcb6-85c8200d8dce None None] MFA Rules not processed for user `9b82c4a811bc44ebab8a6b00f9536634`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:27.069350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-39650f7e-c5a8-497f-bcb6-85c8200d8dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:27.120409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:27.164888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 653/1316] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:31:27.219930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed70adf-cc30-4179-bb77-dc50c02df2b9 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:27.220217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed70adf-cc30-4179-bb77-dc50c02df2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:27.220495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed70adf-cc30-4179-bb77-dc50c02df2b9 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:27.221904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 664/1317] 10.222.0.60 () {54 vars in 965 bytes} [Mon Jul 25 12:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:27.227872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928029ef-476f-4dc1-b313-89820a379bbd None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:27.228155 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928029ef-476f-4dc1-b313-89820a379bbd None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:27.228430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928029ef-476f-4dc1-b313-89820a379bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:27.304144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-928029ef-476f-4dc1-b313-89820a379bbd None None] MFA Rules not processed for user `375d321d6de2494794254e50151c02c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:27.305357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-928029ef-476f-4dc1-b313-89820a379bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:27.379551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 654/1318] 10.222.0.60 () {58 vars in 1062 bytes} [Mon Jul 25 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:31:27.388400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed70adf-cc30-4179-bb77-dc50c02df2b9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:31:27.389944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2ed70adf-cc30-4179-bb77-dc50c02df2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:27.399541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:27.491546 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed70adf-cc30-4179-bb77-dc50c02df2b9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:31:27.562076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a25fe-e991-4ac5-949c-3eedec6873f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:31:27.562715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a25fe-e991-4ac5-949c-3eedec6873f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:27.562994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a25fe-e991-4ac5-949c-3eedec6873f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:27.563267 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a25fe-e991-4ac5-949c-3eedec6873f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:27.565475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5f6a25fe-e991-4ac5-949c-3eedec6873f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:27.640069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:27.686663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a25fe-e991-4ac5-949c-3eedec6873f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:31:27.688115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a25fe-e991-4ac5-949c-3eedec6873f2 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:31:27.706028 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 665/1319] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:31:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:31:30.614439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61fbdf-a9ea-41c0-b961-0c6c5ec04f31 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:30.614972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61fbdf-a9ea-41c0-b961-0c6c5ec04f31 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:30.614972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61fbdf-a9ea-41c0-b961-0c6c5ec04f31 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:30.616384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 655/1320] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:30.621663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa93de-5d0a-4843-b871-64966434ccf7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:30.622121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa93de-5d0a-4843-b871-64966434ccf7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:30.622216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa93de-5d0a-4843-b871-64966434ccf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:30.686368 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-58fa93de-5d0a-4843-b871-64966434ccf7 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:30.687560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-58fa93de-5d0a-4843-b871-64966434ccf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:30.780617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 666/1321] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:31:30.818860 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61fbdf-a9ea-41c0-b961-0c6c5ec04f31 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:31:30.820381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fd61fbdf-a9ea-41c0-b961-0c6c5ec04f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:30.869219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:30.922071 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61fbdf-a9ea-41c0-b961-0c6c5ec04f31 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:31:30.994666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e1e7c-2697-4e9a-82ae-e2c3f725e385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:31:30.995305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e1e7c-2697-4e9a-82ae-e2c3f725e385 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:30.995584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e1e7c-2697-4e9a-82ae-e2c3f725e385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:30.995859 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e1e7c-2697-4e9a-82ae-e2c3f725e385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:30.997481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e80e1e7c-2697-4e9a-82ae-e2c3f725e385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:31.089692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:31.103134 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e1e7c-2697-4e9a-82ae-e2c3f725e385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:31:31.104591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e1e7c-2697-4e9a-82ae-e2c3f725e385 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:31:31.122886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 656/1322] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:31:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:31:33.778594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ff71b-3d53-4380-849b-3a9b55e7161f None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:33.779162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ff71b-3d53-4380-849b-3a9b55e7161f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:33.779162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ff71b-3d53-4380-849b-3a9b55e7161f None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:33.780527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 667/1323] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:33.786140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730694b-845e-4ad1-8f46-73e3f3758e89 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:33.786561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730694b-845e-4ad1-8f46-73e3f3758e89 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:33.786700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730694b-845e-4ad1-8f46-73e3f3758e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:33.852854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-0730694b-845e-4ad1-8f46-73e3f3758e89 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:33.854100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0730694b-845e-4ad1-8f46-73e3f3758e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:33.953797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 657/1324] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:31:33.991382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ff71b-3d53-4380-849b-3a9b55e7161f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:31:33.992951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ce9ff71b-3d53-4380-849b-3a9b55e7161f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:34.013568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:34.107849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ff71b-3d53-4380-849b-3a9b55e7161f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:31:34.189746 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2fec3-bc26-4040-aa07-ad678726a4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:31:34.190432 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2fec3-bc26-4040-aa07-ad678726a4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:34.190710 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2fec3-bc26-4040-aa07-ad678726a4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:34.190979 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2fec3-bc26-4040-aa07-ad678726a4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:34.193242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8dc2fec3-bc26-4040-aa07-ad678726a4c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:34.263646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:34.301615 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2fec3-bc26-4040-aa07-ad678726a4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:31:34.303094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2fec3-bc26-4040-aa07-ad678726a4c3 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:31:34.320756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 668/1325] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:31:36.379110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905fea3-2064-4389-bd63-a854987ea20e None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:36.379659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905fea3-2064-4389-bd63-a854987ea20e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:36.379659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905fea3-2064-4389-bd63-a854987ea20e None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:36.381180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 658/1326] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:36.386390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e0de7-fcd6-4b41-96f2-772a21728859 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:36.386830 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e0de7-fcd6-4b41-96f2-772a21728859 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:36.386927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e0de7-fcd6-4b41-96f2-772a21728859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:36.452792 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-262e0de7-fcd6-4b41-96f2-772a21728859 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:36.453899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-262e0de7-fcd6-4b41-96f2-772a21728859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:36.552316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 669/1327] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:31:36.588829 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905fea3-2064-4389-bd63-a854987ea20e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:31:36.590387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0905fea3-2064-4389-bd63-a854987ea20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:36.650886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:36.694437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905fea3-2064-4389-bd63-a854987ea20e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:31:36.770336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b97fd3-4016-4f0e-9dd9-4fefade66823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:31:36.770990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b97fd3-4016-4f0e-9dd9-4fefade66823 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:36.771274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b97fd3-4016-4f0e-9dd9-4fefade66823 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:36.771554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b97fd3-4016-4f0e-9dd9-4fefade66823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:36.773194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-28b97fd3-4016-4f0e-9dd9-4fefade66823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:36.884026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:36.886473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b97fd3-4016-4f0e-9dd9-4fefade66823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:31:36.888104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b97fd3-4016-4f0e-9dd9-4fefade66823 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:31:36.907806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 659/1328] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:31:38.955202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d36c6-9002-4ae2-b812-752c536230f3 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:38.955726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d36c6-9002-4ae2-b812-752c536230f3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:38.955818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d36c6-9002-4ae2-b812-752c536230f3 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:38.957131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 670/1329] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:38.963016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d59ee-737d-4113-8576-5521ac9e819d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:38.963538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d59ee-737d-4113-8576-5521ac9e819d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:38.963644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d59ee-737d-4113-8576-5521ac9e819d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:39.029019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-160d59ee-737d-4113-8576-5521ac9e819d None None] MFA Rules not processed for user `9b82c4a811bc44ebab8a6b00f9536634`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:39.030289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-160d59ee-737d-4113-8576-5521ac9e819d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:39.128502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 660/1330] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:31:39.165530 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d36c6-9002-4ae2-b812-752c536230f3 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:31:39.167126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-280d36c6-9002-4ae2-b812-752c536230f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:39.201741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:39.277599 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d36c6-9002-4ae2-b812-752c536230f3 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:31:39.348834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1b45a-2b6a-483f-b4c4-e60e0571bd07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:31:39.349475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1b45a-2b6a-483f-b4c4-e60e0571bd07 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:39.349753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1b45a-2b6a-483f-b4c4-e60e0571bd07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:39.350067 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1b45a-2b6a-483f-b4c4-e60e0571bd07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:39.351714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-abc1b45a-2b6a-483f-b4c4-e60e0571bd07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:39.430525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:39.463637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1b45a-2b6a-483f-b4c4-e60e0571bd07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:31:39.465079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1b45a-2b6a-483f-b4c4-e60e0571bd07 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:31:39.483101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 671/1331] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:31:39 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:31:41.756878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ab060-9a6b-4c74-ab07-6cbac92519bb None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:41.757348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ab060-9a6b-4c74-ab07-6cbac92519bb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:41.757463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ab060-9a6b-4c74-ab07-6cbac92519bb None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:41.758881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 661/1332] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:41.763859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bc3fc-2349-4163-9712-1e8962e8b6f3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:41.764297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bc3fc-2349-4163-9712-1e8962e8b6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:41.764394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bc3fc-2349-4163-9712-1e8962e8b6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:41.828670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ca2bc3fc-2349-4163-9712-1e8962e8b6f3 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:41.830400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ca2bc3fc-2349-4163-9712-1e8962e8b6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:41.924755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 672/1333] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:31:41.961934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ab060-9a6b-4c74-ab07-6cbac92519bb None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:31:41.963515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ed4ab060-9a6b-4c74-ab07-6cbac92519bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:42.032724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:42.068453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ab060-9a6b-4c74-ab07-6cbac92519bb None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:31:42.142613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2754d-5a27-4544-81ca-871888727642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:31:42.143256 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2754d-5a27-4544-81ca-871888727642 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:42.143536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2754d-5a27-4544-81ca-871888727642 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:42.143810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2754d-5a27-4544-81ca-871888727642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:42.145432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-97d2754d-5a27-4544-81ca-871888727642 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:42.253017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2754d-5a27-4544-81ca-871888727642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:31:42.254471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2754d-5a27-4544-81ca-871888727642 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:31:42.264121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:42.272471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 662/1334] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:31:44.317973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a1a1-88bf-467b-bc7c-8da9251e4fba None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:44.318498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a1a1-88bf-467b-bc7c-8da9251e4fba None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:44.318498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a1a1-88bf-467b-bc7c-8da9251e4fba None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:44.319912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 673/1335] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:44.325039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611ed1e-2054-44df-80ce-bbcf3f787878 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:44.325453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611ed1e-2054-44df-80ce-bbcf3f787878 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:44.325600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611ed1e-2054-44df-80ce-bbcf3f787878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:44.391311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f611ed1e-2054-44df-80ce-bbcf3f787878 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:44.393052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f611ed1e-2054-44df-80ce-bbcf3f787878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:44.487232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 663/1336] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:31:44.527366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119a1a1-88bf-467b-bc7c-8da9251e4fba None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:31:44.528903 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b119a1a1-88bf-467b-bc7c-8da9251e4fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:44.568982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:44.631409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119a1a1-88bf-467b-bc7c-8da9251e4fba None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:31:44.701714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895ac48-ac27-43e6-a1b3-9e327c544cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:31:44.702393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895ac48-ac27-43e6-a1b3-9e327c544cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:44.702675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895ac48-ac27-43e6-a1b3-9e327c544cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:44.702950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895ac48-ac27-43e6-a1b3-9e327c544cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:44.704589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d895ac48-ac27-43e6-a1b3-9e327c544cf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:44.785012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:44.811129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895ac48-ac27-43e6-a1b3-9e327c544cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:31:44.812568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895ac48-ac27-43e6-a1b3-9e327c544cf3 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:31:44.832277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 674/1337] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:31:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:31:47.293566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a1e19-3123-49ef-8479-1a99d5bd9cd6 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:47.294153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a1e19-3123-49ef-8479-1a99d5bd9cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:47.294153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a1e19-3123-49ef-8479-1a99d5bd9cd6 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:47.295535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 664/1338] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:47.300816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6d9cb-ecc6-4207-b856-77f489c95d6e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:47.301241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6d9cb-ecc6-4207-b856-77f489c95d6e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:47.301375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6d9cb-ecc6-4207-b856-77f489c95d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:47.366370 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b9d6d9cb-ecc6-4207-b856-77f489c95d6e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:47.367586 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b9d6d9cb-ecc6-4207-b856-77f489c95d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:47.462335 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 675/1339] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:31:47.498535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a1e19-3123-49ef-8479-1a99d5bd9cd6 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:31:47.500041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2d5a1e19-3123-49ef-8479-1a99d5bd9cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:47.582641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:47.610758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a1e19-3123-49ef-8479-1a99d5bd9cd6 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:31:47.684460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469f015-9970-4633-b8bc-8f3bdd3c9232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:31:47.685105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469f015-9970-4633-b8bc-8f3bdd3c9232 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:47.685389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469f015-9970-4633-b8bc-8f3bdd3c9232 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:47.685666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469f015-9970-4633-b8bc-8f3bdd3c9232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:47.687346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0469f015-9970-4633-b8bc-8f3bdd3c9232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:47.794319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469f015-9970-4633-b8bc-8f3bdd3c9232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:31:47.795749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469f015-9970-4633-b8bc-8f3bdd3c9232 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:31:47.813316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 665/1340] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:31:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:31:51.767273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1edcc-1a23-493d-9005-7b4af20904b2 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:51.767798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1edcc-1a23-493d-9005-7b4af20904b2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:51.767904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1edcc-1a23-493d-9005-7b4af20904b2 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:51.769341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 676/1341] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:51.774866 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b652f-0086-42dc-85dc-0ec380132aa0 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:51.775376 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b652f-0086-42dc-85dc-0ec380132aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:51.775485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b652f-0086-42dc-85dc-0ec380132aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:51.780885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:51.842419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f35b652f-0086-42dc-85dc-0ec380132aa0 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:51.843629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f35b652f-0086-42dc-85dc-0ec380132aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:51.939892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 666/1342] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:31:51.981176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1edcc-1a23-493d-9005-7b4af20904b2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:31:51.983363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e4f1edcc-1a23-493d-9005-7b4af20904b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:52.032188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:52.087452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1edcc-1a23-493d-9005-7b4af20904b2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:31:52.161057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46442c4-fa1d-4fdc-9d57-5cc0d5e03960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:31:52.161708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46442c4-fa1d-4fdc-9d57-5cc0d5e03960 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:52.162014 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46442c4-fa1d-4fdc-9d57-5cc0d5e03960 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:52.162292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46442c4-fa1d-4fdc-9d57-5cc0d5e03960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:52.163928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c46442c4-fa1d-4fdc-9d57-5cc0d5e03960 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:52.260585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:52.278591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46442c4-fa1d-4fdc-9d57-5cc0d5e03960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:31:52.280046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46442c4-fa1d-4fdc-9d57-5cc0d5e03960 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:31:52.299256 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 677/1343] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:31:54.856004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e74a8-7ec1-476c-b29f-78289f9c5663 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:54.856517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e74a8-7ec1-476c-b29f-78289f9c5663 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:54.856628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e74a8-7ec1-476c-b29f-78289f9c5663 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:54.857940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 667/1344] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:54.863827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d977540-5ce4-4cce-a912-dfd6f40510e2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:54.864227 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d977540-5ce4-4cce-a912-dfd6f40510e2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:54.864379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d977540-5ce4-4cce-a912-dfd6f40510e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:54.939038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1d977540-5ce4-4cce-a912-dfd6f40510e2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:54.940289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1d977540-5ce4-4cce-a912-dfd6f40510e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:55.045170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 678/1345] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:31:55.082998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e74a8-7ec1-476c-b29f-78289f9c5663 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:31:55.084517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e81e74a8-7ec1-476c-b29f-78289f9c5663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:55.187149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:55.210277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e74a8-7ec1-476c-b29f-78289f9c5663 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:31:55.312924 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e2ae4-9dc5-4105-a7ab-a6db4e2274f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:31:55.313857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e2ae4-9dc5-4105-a7ab-a6db4e2274f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:55.314294 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e2ae4-9dc5-4105-a7ab-a6db4e2274f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:55.314678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e2ae4-9dc5-4105-a7ab-a6db4e2274f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:55.317048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9b0e2ae4-9dc5-4105-a7ab-a6db4e2274f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:31:55.463916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e2ae4-9dc5-4105-a7ab-a6db4e2274f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:31:55.466021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e2ae4-9dc5-4105-a7ab-a6db4e2274f8 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:31:55.489966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 668/1346] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:31:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:31:59.884680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ecaea-eee4-4743-80e7-a033d667de40 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:59.885189 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ecaea-eee4-4743-80e7-a033d667de40 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:59.885189 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ecaea-eee4-4743-80e7-a033d667de40 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:59.886777 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 679/1347] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:31:59.893038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cc49d-4ffb-45e5-b5ec-748154f193fc None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:31:59.893545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cc49d-4ffb-45e5-b5ec-748154f193fc None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:31:59.893720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cc49d-4ffb-45e5-b5ec-748154f193fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:31:59.910391 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:31:59.973193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-db3cc49d-4ffb-45e5-b5ec-748154f193fc None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:31:59.974453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-db3cc49d-4ffb-45e5-b5ec-748154f193fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:00.077868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 669/1348] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:00.116792 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ecaea-eee4-4743-80e7-a033d667de40 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:00.118377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5f0ecaea-eee4-4743-80e7-a033d667de40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:00.186000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:00.234323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ecaea-eee4-4743-80e7-a033d667de40 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:00.316141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffde1a-8eb7-453f-984a-51dd4f92dfa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:00.316800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffde1a-8eb7-453f-984a-51dd4f92dfa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:00.317084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffde1a-8eb7-453f-984a-51dd4f92dfa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:00.317360 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffde1a-8eb7-453f-984a-51dd4f92dfa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:00.319067 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d9ffde1a-8eb7-453f-984a-51dd4f92dfa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:00.426860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:00.430911 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffde1a-8eb7-453f-984a-51dd4f92dfa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:00.432361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffde1a-8eb7-453f-984a-51dd4f92dfa5 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:00.451116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 680/1349] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:05.351228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489543f6-206c-44dd-bf3b-b2f803a2bf1f None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:05.351769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489543f6-206c-44dd-bf3b-b2f803a2bf1f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:05.351769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489543f6-206c-44dd-bf3b-b2f803a2bf1f None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:05.353449 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 670/1350] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:05.361173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bf8bc-b24a-428d-b3ca-eb5c72ce79d3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:05.361580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bf8bc-b24a-428d-b3ca-eb5c72ce79d3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:05.362079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bf8bc-b24a-428d-b3ca-eb5c72ce79d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:05.493794 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5a0bf8bc-b24a-428d-b3ca-eb5c72ce79d3 None None] MFA Rules not processed for user `81037f61e01e4a93982503720f7404e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:05.495487 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5a0bf8bc-b24a-428d-b3ca-eb5c72ce79d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:05.643255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 681/1351] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:32:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:09.704760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a555cb-595d-4825-8f18-2f9f60226c23 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:09.705654 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a555cb-595d-4825-8f18-2f9f60226c23 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:09.705963 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a555cb-595d-4825-8f18-2f9f60226c23 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:09.707631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 671/1352] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:32:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:09.712574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0915f57-9d24-40cc-a500-b40bfc6f0b3f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:09.713383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0915f57-9d24-40cc-a500-b40bfc6f0b3f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:09.713901 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0915f57-9d24-40cc-a500-b40bfc6f0b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:09.773354 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:09.799766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c0915f57-9d24-40cc-a500-b40bfc6f0b3f None None] MFA Rules not processed for user `81037f61e01e4a93982503720f7404e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:09.801902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c0915f57-9d24-40cc-a500-b40bfc6f0b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:09.918796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 682/1353] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:10.076186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=59 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:11.174623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d528b-3c6a-4e43-9c3e-fdb028ba3a2c None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:11.175208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d528b-3c6a-4e43-9c3e-fdb028ba3a2c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:11.175208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d528b-3c6a-4e43-9c3e-fdb028ba3a2c None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:11.176528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 672/1354] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:11.181584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f589ea0-3eaa-494a-b066-43df6266e61c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:11.182040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f589ea0-3eaa-494a-b066-43df6266e61c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:11.182226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f589ea0-3eaa-494a-b066-43df6266e61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:11.259525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6f589ea0-3eaa-494a-b066-43df6266e61c None None] MFA Rules not processed for user `81037f61e01e4a93982503720f7404e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:11.260735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6f589ea0-3eaa-494a-b066-43df6266e61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:11.303765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:11.367959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 683/1355] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:32:14.947909 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c15df-b5e2-48cf-b03c-bc3cc6f2a4b2 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:14.949574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c15df-b5e2-48cf-b03c-bc3cc6f2a4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:14.950059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c15df-b5e2-48cf-b03c-bc3cc6f2a4b2 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:14.955793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 673/1356] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:14.957210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d612aa6-990c-434c-89db-f37ee4d7d74d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:14.957628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d612aa6-990c-434c-89db-f37ee4d7d74d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:14.957823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d612aa6-990c-434c-89db-f37ee4d7d74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:15.033729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7d612aa6-990c-434c-89db-f37ee4d7d74d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:15.035345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7d612aa6-990c-434c-89db-f37ee4d7d74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:15.147786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 684/1357] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:15.209626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd0dc3-5323-494c-8342-e4ce51d8cc8e None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:15.210192 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd0dc3-5323-494c-8342-e4ce51d8cc8e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:15.210714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd0dc3-5323-494c-8342-e4ce51d8cc8e None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:15.212527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 674/1358] 10.222.0.60 () {54 vars in 962 bytes} [Mon Jul 25 12:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:15.222740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e71f3-2eb8-4167-8b2a-ccba2cbde5fa None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:15.225133 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e71f3-2eb8-4167-8b2a-ccba2cbde5fa None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:15.225133 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e71f3-2eb8-4167-8b2a-ccba2cbde5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:15.243223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:15.308626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-688e71f3-2eb8-4167-8b2a-ccba2cbde5fa None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:15.309826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-688e71f3-2eb8-4167-8b2a-ccba2cbde5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:15.432656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 685/1359] 10.222.0.60 () {58 vars in 1059 bytes} [Mon Jul 25 12:32:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:15.449700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd0dc3-5323-494c-8342-e4ce51d8cc8e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:15.451477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0cbd0dc3-5323-494c-8342-e4ce51d8cc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:15.608744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd0dc3-5323-494c-8342-e4ce51d8cc8e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:15.696525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:15.740254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e665f-02d6-4d41-8632-d86da44fe336 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:15.741081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e665f-02d6-4d41-8632-d86da44fe336 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:15.741458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e665f-02d6-4d41-8632-d86da44fe336 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:15.741832 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e665f-02d6-4d41-8632-d86da44fe336 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:15.743573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e31e665f-02d6-4d41-8632-d86da44fe336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:15.837935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e665f-02d6-4d41-8632-d86da44fe336 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:15.839970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e665f-02d6-4d41-8632-d86da44fe336 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:15.882055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 675/1360] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:32:18.140990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6adb9-2d86-4967-bf4d-afeb530c6db6 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:18.141882 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6adb9-2d86-4967-bf4d-afeb530c6db6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:18.142214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6adb9-2d86-4967-bf4d-afeb530c6db6 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:18.143822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 686/1361] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:18.149189 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92332a-7c3a-451e-9695-b97298c89ff5 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:18.149636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92332a-7c3a-451e-9695-b97298c89ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:18.149727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92332a-7c3a-451e-9695-b97298c89ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:18.220260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-eb92332a-7c3a-451e-9695-b97298c89ff5 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:18.221451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-eb92332a-7c3a-451e-9695-b97298c89ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:18.260451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:18.334043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 676/1362] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:32:18.376492 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6adb9-2d86-4967-bf4d-afeb530c6db6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:18.377902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-37a6adb9-2d86-4967-bf4d-afeb530c6db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:18.470071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:18.565566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6adb9-2d86-4967-bf4d-afeb530c6db6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:18.715587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d3bf3-63e8-4ffc-8b2b-86af4daea2c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:18.716264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d3bf3-63e8-4ffc-8b2b-86af4daea2c2 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:18.716529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d3bf3-63e8-4ffc-8b2b-86af4daea2c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:18.716804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d3bf3-63e8-4ffc-8b2b-86af4daea2c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:18.718503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-754d3bf3-63e8-4ffc-8b2b-86af4daea2c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:18.742022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:18.811965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d3bf3-63e8-4ffc-8b2b-86af4daea2c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:18.814144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d3bf3-63e8-4ffc-8b2b-86af4daea2c2 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:18.833446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 687/1363] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:21.193857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23694052-e736-4506-a948-61f9b9da0480 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:21.194324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23694052-e736-4506-a948-61f9b9da0480 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:21.194483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23694052-e736-4506-a948-61f9b9da0480 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:21.195950 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 677/1364] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:21.200917 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d35161-eb9e-42dd-93d9-1f1479d9a9d3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:21.201338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d35161-eb9e-42dd-93d9-1f1479d9a9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:21.201488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d35161-eb9e-42dd-93d9-1f1479d9a9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:21.286904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-42d35161-eb9e-42dd-93d9-1f1479d9a9d3 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:21.288158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-42d35161-eb9e-42dd-93d9-1f1479d9a9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:21.393267 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:21.401810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 688/1365] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:21.442411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9666363-6f63-402d-bab1-0b2947b5b279 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:21.442610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9666363-6f63-402d-bab1-0b2947b5b279 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:21.443047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9666363-6f63-402d-bab1-0b2947b5b279 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:21.444405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 678/1366] 10.222.0.60 () {54 vars in 962 bytes} [Mon Jul 25 12:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:21.449713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df3c7f-3321-41b0-9eb6-418917ca47ea None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:21.450458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df3c7f-3321-41b0-9eb6-418917ca47ea None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:21.450717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df3c7f-3321-41b0-9eb6-418917ca47ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:21.532869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e7df3c7f-3321-41b0-9eb6-418917ca47ea None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:21.534149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e7df3c7f-3321-41b0-9eb6-418917ca47ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:21.679416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 689/1367] 10.222.0.60 () {58 vars in 1059 bytes} [Mon Jul 25 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:21.689074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9666363-6f63-402d-bab1-0b2947b5b279 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:21.690699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f9666363-6f63-402d-bab1-0b2947b5b279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:21.876519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9666363-6f63-402d-bab1-0b2947b5b279 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:21.902980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:22.024356 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd37c2-d141-4ebf-b0ab-b170419e3ef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:22.025435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd37c2-d141-4ebf-b0ab-b170419e3ef5 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:22.025886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd37c2-d141-4ebf-b0ab-b170419e3ef5 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:22.026336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd37c2-d141-4ebf-b0ab-b170419e3ef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:22.028134 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-decd37c2-d141-4ebf-b0ab-b170419e3ef5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:22.138040 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd37c2-d141-4ebf-b0ab-b170419e3ef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:22.138691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd37c2-d141-4ebf-b0ab-b170419e3ef5 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:22.160861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 679/1368] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:23.014162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6023058-3a07-4df7-8452-0f6896d3f68f None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:23.014689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6023058-3a07-4df7-8452-0f6896d3f68f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:23.014805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6023058-3a07-4df7-8452-0f6896d3f68f None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:23.020570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 690/1369] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:23.022996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46467366-001d-452c-b49e-e99761dbf63d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:23.023484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46467366-001d-452c-b49e-e99761dbf63d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:23.023589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46467366-001d-452c-b49e-e99761dbf63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:23.074100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:23.123707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-46467366-001d-452c-b49e-e99761dbf63d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:23.124988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-46467366-001d-452c-b49e-e99761dbf63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:23.257673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 680/1370] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:23.632736 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e8a1c-62c3-4050-9f4e-3ae239d81363 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:23.634226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e8a1c-62c3-4050-9f4e-3ae239d81363 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:23.634696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e8a1c-62c3-4050-9f4e-3ae239d81363 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:23.637012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 691/1371] 10.222.0.60 () {54 vars in 964 bytes} [Mon Jul 25 12:32:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:23.645002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c04a2-8834-404c-9abf-c4eda2817181 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:23.645498 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c04a2-8834-404c-9abf-c4eda2817181 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:23.645962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c04a2-8834-404c-9abf-c4eda2817181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:23.733805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-167c04a2-8834-404c-9abf-c4eda2817181 None None] MFA Rules not processed for user `eb4c5027951c4c55bd826943cec15309`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:23.735501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-167c04a2-8834-404c-9abf-c4eda2817181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:23.792974 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:23.834976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 681/1372] 10.222.0.60 () {58 vars in 1061 bytes} [Mon Jul 25 12:32:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:23.842628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e8a1c-62c3-4050-9f4e-3ae239d81363 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:23.844729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f34e8a1c-62c3-4050-9f4e-3ae239d81363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:23.918595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:23.984162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e8a1c-62c3-4050-9f4e-3ae239d81363 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:24.079785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e2d13-8ad0-4438-b1c4-4ce72fb20cea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:24.080448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e2d13-8ad0-4438-b1c4-4ce72fb20cea service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:24.080730 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e2d13-8ad0-4438-b1c4-4ce72fb20cea service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:24.081007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e2d13-8ad0-4438-b1c4-4ce72fb20cea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:24.084004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-303e2d13-8ad0-4438-b1c4-4ce72fb20cea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:24.243534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:24.262121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e2d13-8ad0-4438-b1c4-4ce72fb20cea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:24.263922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e2d13-8ad0-4438-b1c4-4ce72fb20cea service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:24.289186 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 692/1373] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:24.356592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab23202-ab6d-40a2-a93e-db76ff7e8e50 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:24.357044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab23202-ab6d-40a2-a93e-db76ff7e8e50 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:24.357146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab23202-ab6d-40a2-a93e-db76ff7e8e50 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:24.362485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 682/1374] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:24.364524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6da038-e528-4ff2-9535-f828d950cc6f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:24.364811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6da038-e528-4ff2-9535-f828d950cc6f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:24.365085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6da038-e528-4ff2-9535-f828d950cc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:24.452363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ed6da038-e528-4ff2-9535-f828d950cc6f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:24.453767 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ed6da038-e528-4ff2-9535-f828d950cc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:24.580446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 693/1375] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:24.627859 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab23202-ab6d-40a2-a93e-db76ff7e8e50 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:24.629455 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8ab23202-ab6d-40a2-a93e-db76ff7e8e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:24.876135 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab23202-ab6d-40a2-a93e-db76ff7e8e50 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:24.931849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:25.002567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94f646-6e32-4d0d-9665-06e2daafbee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:25.003256 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94f646-6e32-4d0d-9665-06e2daafbee1 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:25.003653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94f646-6e32-4d0d-9665-06e2daafbee1 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:25.003995 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94f646-6e32-4d0d-9665-06e2daafbee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:25.005712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2d94f646-6e32-4d0d-9665-06e2daafbee1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:25.117058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d94f646-6e32-4d0d-9665-06e2daafbee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:25.118781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d94f646-6e32-4d0d-9665-06e2daafbee1 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:25.139501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 683/1376] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:27.299757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f2684-e661-40b5-a866-4ba50458cdf8 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:27.300257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f2684-e661-40b5-a866-4ba50458cdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:27.300338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f2684-e661-40b5-a866-4ba50458cdf8 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:27.301826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 694/1377] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:27.307281 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350562a-545d-47aa-b414-cecfff0b4a74 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:27.307745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350562a-545d-47aa-b414-cecfff0b4a74 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:27.307839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350562a-545d-47aa-b414-cecfff0b4a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:27.377191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f350562a-545d-47aa-b414-cecfff0b4a74 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:27.378942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f350562a-545d-47aa-b414-cecfff0b4a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:27.385660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:27.486281 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 684/1378] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:27.532345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f2684-e661-40b5-a866-4ba50458cdf8 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:27.533881 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-641f2684-e661-40b5-a866-4ba50458cdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:27.611435 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:27.736761 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f2684-e661-40b5-a866-4ba50458cdf8 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:27.871043 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7324986-900b-4026-bdd7-fafb49bafc78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:27.871703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7324986-900b-4026-bdd7-fafb49bafc78 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:27.871988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7324986-900b-4026-bdd7-fafb49bafc78 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:27.872266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7324986-900b-4026-bdd7-fafb49bafc78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:27.873899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a7324986-900b-4026-bdd7-fafb49bafc78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:27.895414 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:27.983082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7324986-900b-4026-bdd7-fafb49bafc78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:27.986077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7324986-900b-4026-bdd7-fafb49bafc78 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:28.006860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 695/1379] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:30.133773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1949a-01b6-419d-92df-d46ad0712269 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:30.134346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1949a-01b6-419d-92df-d46ad0712269 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:30.134615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1949a-01b6-419d-92df-d46ad0712269 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:30.136459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 685/1380] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:30.141703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e012c-1baa-4cfc-9414-3ee34134ab4e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:30.142666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e012c-1baa-4cfc-9414-3ee34134ab4e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:30.142952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e012c-1baa-4cfc-9414-3ee34134ab4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:30.220881 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7a8e012c-1baa-4cfc-9414-3ee34134ab4e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:30.222021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7a8e012c-1baa-4cfc-9414-3ee34134ab4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:30.322091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:30.349994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 696/1381] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:30.375106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1949a-01b6-419d-92df-d46ad0712269 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:30.376807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6fc1949a-01b6-419d-92df-d46ad0712269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:30.549633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:30.563755 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1949a-01b6-419d-92df-d46ad0712269 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:30.724714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c839d-b428-4f68-9f59-7a9797c0e418 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:30.725455 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c839d-b428-4f68-9f59-7a9797c0e418 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:30.725935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c839d-b428-4f68-9f59-7a9797c0e418 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:30.726315 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c839d-b428-4f68-9f59-7a9797c0e418 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:30.728031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-429c839d-b428-4f68-9f59-7a9797c0e418 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:30.975018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c839d-b428-4f68-9f59-7a9797c0e418 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:30.982759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c839d-b428-4f68-9f59-7a9797c0e418 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:31.012190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:31.036527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 686/1382] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:33.237747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac1508-ad92-49ee-8ade-d5e0b5f2267d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:33.238554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac1508-ad92-49ee-8ade-d5e0b5f2267d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:33.238554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac1508-ad92-49ee-8ade-d5e0b5f2267d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:33.239721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 697/1383] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:33.245703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e52b31-46d5-4584-8598-7b5bdd710539 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:33.246191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e52b31-46d5-4584-8598-7b5bdd710539 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:33.246552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e52b31-46d5-4584-8598-7b5bdd710539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:33.342802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-38e52b31-46d5-4584-8598-7b5bdd710539 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:33.344849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-38e52b31-46d5-4584-8598-7b5bdd710539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:33.453362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 687/1384] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:33.496614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac1508-ad92-49ee-8ade-d5e0b5f2267d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:33.498620 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-15ac1508-ad92-49ee-8ade-d5e0b5f2267d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:33.771253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac1508-ad92-49ee-8ade-d5e0b5f2267d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:33.852782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:33.903337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8eff6-c0f4-4d12-a57a-e560b228cfd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:33.904141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8eff6-c0f4-4d12-a57a-e560b228cfd4 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:33.904439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8eff6-c0f4-4d12-a57a-e560b228cfd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:33.904729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8eff6-c0f4-4d12-a57a-e560b228cfd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:33.906514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-aed8eff6-c0f4-4d12-a57a-e560b228cfd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:34.007658 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8eff6-c0f4-4d12-a57a-e560b228cfd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:34.009723 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8eff6-c0f4-4d12-a57a-e560b228cfd4 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:34.032405 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 698/1385] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:36.130349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2d5fe-d050-4627-8564-3a416ca43d2b None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:36.130892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2d5fe-d050-4627-8564-3a416ca43d2b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:36.131141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2d5fe-d050-4627-8564-3a416ca43d2b None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:36.133091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 688/1386] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:36.138355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853c2c8-7400-4b9b-b6c1-cf45b1734de7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:36.139069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853c2c8-7400-4b9b-b6c1-cf45b1734de7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:36.139201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853c2c8-7400-4b9b-b6c1-cf45b1734de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:36.212387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7853c2c8-7400-4b9b-b6c1-cf45b1734de7 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:36.213541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7853c2c8-7400-4b9b-b6c1-cf45b1734de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:36.248026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:36.323570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 699/1387] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:36.372633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2d5fe-d050-4627-8564-3a416ca43d2b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:36.374839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0cd2d5fe-d050-4627-8564-3a416ca43d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:36.481517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:36.580841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2d5fe-d050-4627-8564-3a416ca43d2b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:36.703275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ab63f-346f-4025-b95c-e8d0c01bab59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:36.703931 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ab63f-346f-4025-b95c-e8d0c01bab59 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:36.704217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ab63f-346f-4025-b95c-e8d0c01bab59 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:36.704543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ab63f-346f-4025-b95c-e8d0c01bab59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:36.706226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-936ab63f-346f-4025-b95c-e8d0c01bab59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:36.756385 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:36.828701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ab63f-346f-4025-b95c-e8d0c01bab59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:36.830364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ab63f-346f-4025-b95c-e8d0c01bab59 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:36.848304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 689/1388] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:39.192911 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a00978-36f8-44a4-8f60-17f14d93db2d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:39.193385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a00978-36f8-44a4-8f60-17f14d93db2d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:39.193511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a00978-36f8-44a4-8f60-17f14d93db2d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:39.194848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 700/1389] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:39.199955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495281e-c67d-4b09-bdc7-5c211a9bb3fe None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:39.200367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495281e-c67d-4b09-bdc7-5c211a9bb3fe None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:39.200516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495281e-c67d-4b09-bdc7-5c211a9bb3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:39.266825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-9495281e-c67d-4b09-bdc7-5c211a9bb3fe None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:39.268021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9495281e-c67d-4b09-bdc7-5c211a9bb3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:39.363060 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 690/1390] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:39.407204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a00978-36f8-44a4-8f60-17f14d93db2d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:39.408704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-47a00978-36f8-44a4-8f60-17f14d93db2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:39.554161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a00978-36f8-44a4-8f60-17f14d93db2d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:39.571101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:39.667387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f44d74-d499-4a28-b11a-f7701656972c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:39.668036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f44d74-d499-4a28-b11a-f7701656972c service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:39.668314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f44d74-d499-4a28-b11a-f7701656972c service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:39.668583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f44d74-d499-4a28-b11a-f7701656972c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:39.670241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-14f44d74-d499-4a28-b11a-f7701656972c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:39.757356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f44d74-d499-4a28-b11a-f7701656972c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:39.758992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f44d74-d499-4a28-b11a-f7701656972c service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:39.776508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 701/1391] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:40.067608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af91837-0c8a-4e31-99ae-66829b8295d0 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:40.067916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af91837-0c8a-4e31-99ae-66829b8295d0 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:40.068198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af91837-0c8a-4e31-99ae-66829b8295d0 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:40.069571 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 691/1392] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:40.074866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c609e-c4b6-4f72-bf7f-667dc82b4416 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:40.075160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c609e-c4b6-4f72-bf7f-667dc82b4416 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:40.075440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c609e-c4b6-4f72-bf7f-667dc82b4416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:40.102140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:40.144609 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-fe2c609e-c4b6-4f72-bf7f-667dc82b4416 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:40.145814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fe2c609e-c4b6-4f72-bf7f-667dc82b4416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:40.244936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 702/1393] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:40.300745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47674784-eb8d-47a7-b393-cf90adb93911 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:40.301173 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47674784-eb8d-47a7-b393-cf90adb93911 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:40.301326 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47674784-eb8d-47a7-b393-cf90adb93911 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:40.302744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 692/1394] 10.222.0.60 () {54 vars in 966 bytes} [Mon Jul 25 12:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:40.307896 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51eb8b6-7083-4e34-941a-1ac42111ae66 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:40.308254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51eb8b6-7083-4e34-941a-1ac42111ae66 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:40.308602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51eb8b6-7083-4e34-941a-1ac42111ae66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:40.384938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f51eb8b6-7083-4e34-941a-1ac42111ae66 None None] MFA Rules not processed for user `6b07dcd8c0de4d1694d4363520726df2`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:40.386079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f51eb8b6-7083-4e34-941a-1ac42111ae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:40.425966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:40.462421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 703/1395] 10.222.0.60 () {58 vars in 1063 bytes} [Mon Jul 25 12:32:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:32:40.469997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47674784-eb8d-47a7-b393-cf90adb93911 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:40.471643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-47674784-eb8d-47a7-b393-cf90adb93911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:40.485638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:40.575653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47674784-eb8d-47a7-b393-cf90adb93911 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:40.649253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7076d78-b0db-4511-8570-18f17d62926f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b07dcd8c0de4d1694d4363520726df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:40.649808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7076d78-b0db-4511-8570-18f17d62926f service glance] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:40.650111 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7076d78-b0db-4511-8570-18f17d62926f service glance] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:40.650383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7076d78-b0db-4511-8570-18f17d62926f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:40.652010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f7076d78-b0db-4511-8570-18f17d62926f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:40.720969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:40.762013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7076d78-b0db-4511-8570-18f17d62926f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:40.763444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7076d78-b0db-4511-8570-18f17d62926f service glance] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:40.782058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 693/1396] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:41.513277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc809958-0e51-41ff-a958-d5aff9c1afcf None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:41.513766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc809958-0e51-41ff-a958-d5aff9c1afcf None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:41.514008 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc809958-0e51-41ff-a958-d5aff9c1afcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:41.579900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-fc809958-0e51-41ff-a958-d5aff9c1afcf None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:41.581105 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fc809958-0e51-41ff-a958-d5aff9c1afcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:41.668769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 704/1397] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:32:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:41.711979 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26096b-a963-4ed5-b86b-972e3fe65842 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:41.712469 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26096b-a963-4ed5-b86b-972e3fe65842 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:41.712565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26096b-a963-4ed5-b86b-972e3fe65842 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:41.713928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 694/1398] 10.222.0.60 () {54 vars in 943 bytes} [Mon Jul 25 12:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:41.721335 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4beb813-32d1-4431-91b1-b935843f5aef None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:41.721646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4beb813-32d1-4431-91b1-b935843f5aef None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:41.721966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4beb813-32d1-4431-91b1-b935843f5aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:41.788710 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:41.798274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d4beb813-32d1-4431-91b1-b935843f5aef None None] MFA Rules not processed for user `e56b0d48e08c4f778a7c509c10edc871`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:41.799493 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d4beb813-32d1-4431-91b1-b935843f5aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:41.812904 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e42ff-d386-4685-9d64-c6b51afb2ea3 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:41.813272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e42ff-d386-4685-9d64-c6b51afb2ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:41.813567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e42ff-d386-4685-9d64-c6b51afb2ea3 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:41.814998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 695/1399] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:32:41.820412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c07ec6-9c9a-46ee-8b99-1c0e773530fb None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:41.820694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c07ec6-9c9a-46ee-8b99-1c0e773530fb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:41.820970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c07ec6-9c9a-46ee-8b99-1c0e773530fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:41.886932 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-91c07ec6-9c9a-46ee-8b99-1c0e773530fb None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:41.888146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-91c07ec6-9c9a-46ee-8b99-1c0e773530fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:41.907612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 705/1400] 10.222.0.60 () {58 vars in 1040 bytes} [Mon Jul 25 12:32:41 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:41.917036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4beb813-32d1-4431-91b1-b935843f5aef None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:41.926936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4beb813-32d1-4431-91b1-b935843f5aef None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:41.988200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 696/1401] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:32:42.031978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10ea51-ca0c-4da6-b934-cfd38fe69eb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:42.032636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c07ec6-9c9a-46ee-8b99-1c0e773530fb None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:42.032769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10ea51-ca0c-4da6-b934-cfd38fe69eb6 service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:42.032937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10ea51-ca0c-4da6-b934-cfd38fe69eb6 service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:42.033208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10ea51-ca0c-4da6-b934-cfd38fe69eb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:42.033816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-91c07ec6-9c9a-46ee-8b99-1c0e773530fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:42.034912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2c10ea51-ca0c-4da6-b934-cfd38fe69eb6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:42.037513 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:42.062066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:42.124292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10ea51-ca0c-4da6-b934-cfd38fe69eb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:42.125956 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10ea51-ca0c-4da6-b934-cfd38fe69eb6 service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:42.134351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 706/1402] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:32:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 12:32:42.152224 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c07ec6-9c9a-46ee-8b99-1c0e773530fb None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:42.279389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d33fff-a9aa-4c34-ad69-754fae94aba7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:42.280679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d33fff-a9aa-4c34-ad69-754fae94aba7 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:42.280967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d33fff-a9aa-4c34-ad69-754fae94aba7 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:42.281244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d33fff-a9aa-4c34-ad69-754fae94aba7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:42.282958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d5d33fff-a9aa-4c34-ad69-754fae94aba7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:42.301031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:42.386024 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d33fff-a9aa-4c34-ad69-754fae94aba7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:42.387879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d33fff-a9aa-4c34-ad69-754fae94aba7 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:42.410465 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 697/1403] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:32:44.474185 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32361e5-42be-4b35-abdc-e8215b2df0d5 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:44.474753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32361e5-42be-4b35-abdc-e8215b2df0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:44.474753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32361e5-42be-4b35-abdc-e8215b2df0d5 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:44.476006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 707/1404] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:44.481466 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6ccc7-86f1-461a-bc5a-147aa1200a39 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:44.481878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6ccc7-86f1-461a-bc5a-147aa1200a39 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:44.482093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6ccc7-86f1-461a-bc5a-147aa1200a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:44.553677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-32c6ccc7-86f1-461a-bc5a-147aa1200a39 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:44.554922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-32c6ccc7-86f1-461a-bc5a-147aa1200a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:44.661650 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:44.669842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 698/1405] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:32:44.713972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32361e5-42be-4b35-abdc-e8215b2df0d5 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:44.715535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c32361e5-42be-4b35-abdc-e8215b2df0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:44.874751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:44.883168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32361e5-42be-4b35-abdc-e8215b2df0d5 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:45.010681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd536225-8e84-41fd-9991-98546714e7fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:45.011432 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd536225-8e84-41fd-9991-98546714e7fc service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:45.011783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd536225-8e84-41fd-9991-98546714e7fc service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:45.012120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd536225-8e84-41fd-9991-98546714e7fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:45.013822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cd536225-8e84-41fd-9991-98546714e7fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:45.113322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd536225-8e84-41fd-9991-98546714e7fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:45.114867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd536225-8e84-41fd-9991-98546714e7fc service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:45.133129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 708/1406] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:47.194920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6cb68-d4c3-4f19-98cf-a35c852499e3 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:47.195418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6cb68-d4c3-4f19-98cf-a35c852499e3 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:47.195537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6cb68-d4c3-4f19-98cf-a35c852499e3 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:47.197014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 699/1407] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:32:47.202199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783d709-f949-42f9-9803-b61d0d53fb2b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:47.202638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783d709-f949-42f9-9803-b61d0d53fb2b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:47.202776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783d709-f949-42f9-9803-b61d0d53fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:47.207627 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:47.278495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e783d709-f949-42f9-9803-b61d0d53fb2b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:32:47.280287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e783d709-f949-42f9-9803-b61d0d53fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:47.402297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 709/1408] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:32:47.436522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6cb68-d4c3-4f19-98cf-a35c852499e3 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:47.438129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-46a6cb68-d4c3-4f19-98cf-a35c852499e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:47.605539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6cb68-d4c3-4f19-98cf-a35c852499e3 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:47.692599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:47.770628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151ebec-4ce6-4944-9a24-895fdb41e87a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:47.773186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151ebec-4ce6-4944-9a24-895fdb41e87a service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:47.773943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151ebec-4ce6-4944-9a24-895fdb41e87a service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:47.774232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151ebec-4ce6-4944-9a24-895fdb41e87a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:47.776034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8151ebec-4ce6-4944-9a24-895fdb41e87a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:47.893033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151ebec-4ce6-4944-9a24-895fdb41e87a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:47.894773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151ebec-4ce6-4944-9a24-895fdb41e87a service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:47.915469 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 700/1409] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:32:57.388284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=64 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:57.912840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=64 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:04.252013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611a04d-4fbc-4d56-990c-16a018d8044b None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:04.252583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611a04d-4fbc-4d56-990c-16a018d8044b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:04.252583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611a04d-4fbc-4d56-990c-16a018d8044b None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:04.253997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 710/1410] 10.222.0.60 () {54 vars in 918 bytes} [Mon Jul 25 12:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:04.259345 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9a78c-a8ac-4708-9e51-44612cf45ee8 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:04.259770 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9a78c-a8ac-4708-9e51-44612cf45ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:04.259900 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9a78c-a8ac-4708-9e51-44612cf45ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:04.335444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-02e9a78c-a8ac-4708-9e51-44612cf45ee8 None None] MFA Rules not processed for user `81037f61e01e4a93982503720f7404e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:04.336638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-02e9a78c-a8ac-4708-9e51-44612cf45ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:04.440263 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 701/1411] 10.222.0.60 () {58 vars in 1015 bytes} [Mon Jul 25 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:04.454917 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdcad8-00b7-4062-8d41-4f1e04630425 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:04.455210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdcad8-00b7-4062-8d41-4f1e04630425 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:04.455493 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdcad8-00b7-4062-8d41-4f1e04630425 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:04.456885 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 711/1412] 10.222.0.60 () {54 vars in 918 bytes} [Mon Jul 25 12:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:04.461329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700f119-1ea6-4480-8fc7-baa52135868a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:04.461620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700f119-1ea6-4480-8fc7-baa52135868a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:04.461893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700f119-1ea6-4480-8fc7-baa52135868a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:04.532688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:04.536001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-2700f119-1ea6-4480-8fc7-baa52135868a None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:04.537723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2700f119-1ea6-4480-8fc7-baa52135868a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:04.650464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 702/1413] 10.222.0.60 () {58 vars in 1015 bytes} [Mon Jul 25 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:07.108824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b940e-8477-461d-9d04-5218eed8c277 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:07.109378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b940e-8477-461d-9d04-5218eed8c277 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:07.109378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b940e-8477-461d-9d04-5218eed8c277 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:07.110789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:33:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:07.116130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48994ff-dfd6-4063-b48c-2cb0171697b2 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:07.116556 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48994ff-dfd6-4063-b48c-2cb0171697b2 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:07.116653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48994ff-dfd6-4063-b48c-2cb0171697b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:07.182079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-e48994ff-dfd6-4063-b48c-2cb0171697b2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:07.183288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e48994ff-dfd6-4063-b48c-2cb0171697b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:07.237821 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:07.281095 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:07.324251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b940e-8477-461d-9d04-5218eed8c277 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:07.325864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d05b940e-8477-461d-9d04-5218eed8c277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:07.468244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b940e-8477-461d-9d04-5218eed8c277 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:07.538594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:07.579482 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacd6da-32bd-40ff-85cc-091259553905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:07.580141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacd6da-32bd-40ff-85cc-091259553905 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:07.580421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacd6da-32bd-40ff-85cc-091259553905 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:07.580694 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacd6da-32bd-40ff-85cc-091259553905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:07.582375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-eeacd6da-32bd-40ff-85cc-091259553905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:07.672741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacd6da-32bd-40ff-85cc-091259553905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:07.674424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacd6da-32bd-40ff-85cc-091259553905 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:07.692132 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 713/1416] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:08.594369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e98cc-4048-45cf-a6f4-72db484b2195 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:08.594902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e98cc-4048-45cf-a6f4-72db484b2195 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:08.594902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e98cc-4048-45cf-a6f4-72db484b2195 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:08.596291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:08.601154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669fcac-e8a0-474f-b36d-fc08a4119462 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:08.601547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669fcac-e8a0-474f-b36d-fc08a4119462 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:08.601705 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669fcac-e8a0-474f-b36d-fc08a4119462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:08.676198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4669fcac-e8a0-474f-b36d-fc08a4119462 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:08.677401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4669fcac-e8a0-474f-b36d-fc08a4119462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:08.703904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:08.772140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:33:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:08.926723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e98cc-4048-45cf-a6f4-72db484b2195 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:08.928255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7b2e98cc-4048-45cf-a6f4-72db484b2195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:09.070413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e98cc-4048-45cf-a6f4-72db484b2195 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:09.109091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:09.180817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4765185-fb14-490e-9542-fe776805e47b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:09.181451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4765185-fb14-490e-9542-fe776805e47b service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:09.181733 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4765185-fb14-490e-9542-fe776805e47b service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:09.182036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4765185-fb14-490e-9542-fe776805e47b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:09.183713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d4765185-fb14-490e-9542-fe776805e47b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:09.271469 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4765185-fb14-490e-9542-fe776805e47b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:09.273061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4765185-fb14-490e-9542-fe776805e47b service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:09.290639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 705/1419] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:10.165752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b85033-e007-47eb-8b9a-6512c27bc6aa None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:10.166356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b85033-e007-47eb-8b9a-6512c27bc6aa None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:10.166503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b85033-e007-47eb-8b9a-6512c27bc6aa None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:10.167924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:10.173042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121f2e8-6a7f-4c82-90be-cb0389a0e04b None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:10.173459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121f2e8-6a7f-4c82-90be-cb0389a0e04b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:10.173587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121f2e8-6a7f-4c82-90be-cb0389a0e04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:10.226769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:10.251186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-6121f2e8-6a7f-4c82-90be-cb0389a0e04b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:10.252406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6121f2e8-6a7f-4c82-90be-cb0389a0e04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:10.348129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:10.496863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b85033-e007-47eb-8b9a-6512c27bc6aa None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:10.498906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-23b85033-e007-47eb-8b9a-6512c27bc6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:10.642937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b85033-e007-47eb-8b9a-6512c27bc6aa None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:10.654914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:10.755559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b3752-6c3b-4e08-ab84-1741aa0a1b97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:10.756132 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b3752-6c3b-4e08-ab84-1741aa0a1b97 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:10.756416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b3752-6c3b-4e08-ab84-1741aa0a1b97 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:10.756695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b3752-6c3b-4e08-ab84-1741aa0a1b97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:10.758370 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-500b3752-6c3b-4e08-ab84-1741aa0a1b97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:10.856314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b3752-6c3b-4e08-ab84-1741aa0a1b97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:10.858006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b3752-6c3b-4e08-ab84-1741aa0a1b97 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:10.876315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 716/1422] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:11.727681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30996f3-b649-483b-a03e-08ca20e0c95d None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:11.728623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30996f3-b649-483b-a03e-08ca20e0c95d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:11.728623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30996f3-b649-483b-a03e-08ca20e0c95d None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:11.729861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:11.735254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795c6fc-c6c8-485a-b4e7-1a0fe2abf17d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:11.735704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795c6fc-c6c8-485a-b4e7-1a0fe2abf17d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:11.735830 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795c6fc-c6c8-485a-b4e7-1a0fe2abf17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:11.755849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:11.811212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d795c6fc-c6c8-485a-b4e7-1a0fe2abf17d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:11.812402 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d795c6fc-c6c8-485a-b4e7-1a0fe2abf17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:11.907738 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:12.059303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30996f3-b649-483b-a03e-08ca20e0c95d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:12.061338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b30996f3-b649-483b-a03e-08ca20e0c95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:12.169271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:12.206131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30996f3-b649-483b-a03e-08ca20e0c95d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:12.325278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b0750-757a-4821-9b09-9beea45cb186 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:12.325865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b0750-757a-4821-9b09-9beea45cb186 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:12.326175 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b0750-757a-4821-9b09-9beea45cb186 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:12.326443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b0750-757a-4821-9b09-9beea45cb186 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:12.328070 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c36b0750-757a-4821-9b09-9beea45cb186 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:12.401597 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:12.415008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b0750-757a-4821-9b09-9beea45cb186 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:12.416601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b0750-757a-4821-9b09-9beea45cb186 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:12.434433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 708/1425] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:13.289590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553b713-9ce3-4bb8-845d-5c577290c4e4 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:13.290142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553b713-9ce3-4bb8-845d-5c577290c4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:13.290390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553b713-9ce3-4bb8-845d-5c577290c4e4 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:13.291863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:33:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:13.297470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb469a0-5781-4afe-ac2b-c3a8b57d5041 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:13.297941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb469a0-5781-4afe-ac2b-c3a8b57d5041 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:13.298041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb469a0-5781-4afe-ac2b-c3a8b57d5041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:13.372450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-4bb469a0-5781-4afe-ac2b-c3a8b57d5041 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:13.373653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4bb469a0-5781-4afe-ac2b-c3a8b57d5041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:13.467654 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:13.617526 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553b713-9ce3-4bb8-845d-5c577290c4e4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:13.619121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0553b713-9ce3-4bb8-845d-5c577290c4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:13.701156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:13.761600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553b713-9ce3-4bb8-845d-5c577290c4e4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:13.871029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d573785-b386-48f1-8f56-9626e87b2f00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:13.871711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d573785-b386-48f1-8f56-9626e87b2f00 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:13.871992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d573785-b386-48f1-8f56-9626e87b2f00 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:13.872322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d573785-b386-48f1-8f56-9626e87b2f00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:13.874072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8d573785-b386-48f1-8f56-9626e87b2f00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:13.922635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:13.964800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d573785-b386-48f1-8f56-9626e87b2f00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:13.967045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d573785-b386-48f1-8f56-9626e87b2f00 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:13.985026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 719/1428] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:14.796899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83ae37-a055-47c6-b177-d5acfdcc299b None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:14.797467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83ae37-a055-47c6-b177-d5acfdcc299b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:14.797467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83ae37-a055-47c6-b177-d5acfdcc299b None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:14.798930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:14.803888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec0bd4-25c7-4b6d-8adc-5a9210048449 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:14.804334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec0bd4-25c7-4b6d-8adc-5a9210048449 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:14.804518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec0bd4-25c7-4b6d-8adc-5a9210048449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:14.879686 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1aec0bd4-25c7-4b6d-8adc-5a9210048449 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:14.880892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1aec0bd4-25c7-4b6d-8adc-5a9210048449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:14.974990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:15.125675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83ae37-a055-47c6-b177-d5acfdcc299b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:15.127194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4a83ae37-a055-47c6-b177-d5acfdcc299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:15.175155 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:15.269765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83ae37-a055-47c6-b177-d5acfdcc299b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:15.378529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b28de-81c6-4bfd-ad11-566b00013f45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:15.379107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b28de-81c6-4bfd-ad11-566b00013f45 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:15.379418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b28de-81c6-4bfd-ad11-566b00013f45 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:15.379697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b28de-81c6-4bfd-ad11-566b00013f45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:15.381327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4b4b28de-81c6-4bfd-ad11-566b00013f45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:15.388663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:15.471267 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b28de-81c6-4bfd-ad11-566b00013f45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:15.472895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b28de-81c6-4bfd-ad11-566b00013f45 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:15.491209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 711/1431] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:16.290683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66b23f-406d-4b49-b6ab-dabb39b557aa None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:16.291240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66b23f-406d-4b49-b6ab-dabb39b557aa None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:16.291240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66b23f-406d-4b49-b6ab-dabb39b557aa None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:16.292673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:16.298106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2f6bc-e055-4996-968b-3a02131498e5 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:16.298545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2f6bc-e055-4996-968b-3a02131498e5 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:16.298697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2f6bc-e055-4996-968b-3a02131498e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:16.373740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-7af2f6bc-e055-4996-968b-3a02131498e5 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:16.374975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7af2f6bc-e055-4996-968b-3a02131498e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:16.441394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:16.471304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:16.622087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66b23f-406d-4b49-b6ab-dabb39b557aa None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:16.623648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7e66b23f-406d-4b49-b6ab-dabb39b557aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:16.640831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:16.769679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66b23f-406d-4b49-b6ab-dabb39b557aa None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:16.856167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:16.881438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7b439-0b39-480e-ab07-1b6f2e1813d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:16.882116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7b439-0b39-480e-ab07-1b6f2e1813d6 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:16.882396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7b439-0b39-480e-ab07-1b6f2e1813d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:16.882664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7b439-0b39-480e-ab07-1b6f2e1813d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:16.884289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2da7b439-0b39-480e-ab07-1b6f2e1813d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:16.974167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7b439-0b39-480e-ab07-1b6f2e1813d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:16.975760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7b439-0b39-480e-ab07-1b6f2e1813d6 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:16.993599 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 722/1434] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:17.789767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6ee74-a269-48aa-b09a-e8bc60150e53 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:17.790369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6ee74-a269-48aa-b09a-e8bc60150e53 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:17.790369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6ee74-a269-48aa-b09a-e8bc60150e53 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:17.791708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:17.797104 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6882f6-48a6-41c9-aaad-3575530c3e8d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:17.797485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6882f6-48a6-41c9-aaad-3575530c3e8d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:17.797684 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6882f6-48a6-41c9-aaad-3575530c3e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:17.875175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5e6882f6-48a6-41c9-aaad-3575530c3e8d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:17.876385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5e6882f6-48a6-41c9-aaad-3575530c3e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:17.916928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:17.972680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:18.124489 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6ee74-a269-48aa-b09a-e8bc60150e53 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:18.125935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c6f6ee74-a269-48aa-b09a-e8bc60150e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:18.269663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6ee74-a269-48aa-b09a-e8bc60150e53 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:18.322115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:18.380722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90d042-b96c-4004-abca-15aa5b832988 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:18.381285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90d042-b96c-4004-abca-15aa5b832988 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:18.381563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90d042-b96c-4004-abca-15aa5b832988 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:18.381834 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90d042-b96c-4004-abca-15aa5b832988 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:18.383496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2c90d042-b96c-4004-abca-15aa5b832988 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:18.470696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90d042-b96c-4004-abca-15aa5b832988 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:18.472288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90d042-b96c-4004-abca-15aa5b832988 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:18.490131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 714/1437] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:19.290551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ac389-6cbb-4802-8b20-8c89754b0b7f None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:19.291097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ac389-6cbb-4802-8b20-8c89754b0b7f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:19.291097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ac389-6cbb-4802-8b20-8c89754b0b7f None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:19.292473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:19.297837 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a763714-3439-4f95-be43-d294598af2e0 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:19.298275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a763714-3439-4f95-be43-d294598af2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:19.298428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a763714-3439-4f95-be43-d294598af2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:19.373090 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-9a763714-3439-4f95-be43-d294598af2e0 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:19.374360 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9a763714-3439-4f95-be43-d294598af2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:19.376356 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:19.471719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:19.624153 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ac389-6cbb-4802-8b20-8c89754b0b7f None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:19.625660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2b0ac389-6cbb-4802-8b20-8c89754b0b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:19.769102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ac389-6cbb-4802-8b20-8c89754b0b7f None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:19.795698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:19.881676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540af5ad-a0d5-4a84-b9f2-9dcdc6972e1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:19.882365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540af5ad-a0d5-4a84-b9f2-9dcdc6972e1a service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:19.882653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540af5ad-a0d5-4a84-b9f2-9dcdc6972e1a service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:19.882932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540af5ad-a0d5-4a84-b9f2-9dcdc6972e1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:19.884583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-540af5ad-a0d5-4a84-b9f2-9dcdc6972e1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:19.975136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540af5ad-a0d5-4a84-b9f2-9dcdc6972e1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:19.976730 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540af5ad-a0d5-4a84-b9f2-9dcdc6972e1a service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:19.994823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 725/1440] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:21.900916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d038e74-14d1-474b-8e59-4b5c00f23c27 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:21.901483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d038e74-14d1-474b-8e59-4b5c00f23c27 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:21.901483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d038e74-14d1-474b-8e59-4b5c00f23c27 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:21.902873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:21.908696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266412d2-0bc5-4c05-aa60-3bda1556c610 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:21.909154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266412d2-0bc5-4c05-aa60-3bda1556c610 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:21.909293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266412d2-0bc5-4c05-aa60-3bda1556c610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:21.949311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:21.974746 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-266412d2-0bc5-4c05-aa60-3bda1556c610 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:21.975936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-266412d2-0bc5-4c05-aa60-3bda1556c610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:22.070255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:22.109226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d038e74-14d1-474b-8e59-4b5c00f23c27 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:22.110799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9d038e74-14d1-474b-8e59-4b5c00f23c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:22.215497 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d038e74-14d1-474b-8e59-4b5c00f23c27 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:22.247498 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:22.289858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23256b-feda-4515-b01e-95554b071dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:22.290528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23256b-feda-4515-b01e-95554b071dfc service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:22.290809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23256b-feda-4515-b01e-95554b071dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:22.291083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23256b-feda-4515-b01e-95554b071dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:22.292710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2d23256b-feda-4515-b01e-95554b071dfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:22.401161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23256b-feda-4515-b01e-95554b071dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:22.402625 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23256b-feda-4515-b01e-95554b071dfc service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:22.420754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 717/1443] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:24.577808 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b286703-a414-444b-b837-26730da69797 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:24.578443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b286703-a414-444b-b837-26730da69797 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:24.578443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b286703-a414-444b-b837-26730da69797 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:24.579833 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:24.584975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bee372-c10f-4715-b1da-5fdad90dc7bc None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:24.585370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bee372-c10f-4715-b1da-5fdad90dc7bc None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:24.585533 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bee372-c10f-4715-b1da-5fdad90dc7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:24.649873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-a9bee372-c10f-4715-b1da-5fdad90dc7bc None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:24.651097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a9bee372-c10f-4715-b1da-5fdad90dc7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:24.664025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:24.746098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:24.784513 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b286703-a414-444b-b837-26730da69797 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:24.786095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4b286703-a414-444b-b837-26730da69797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:24.893158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b286703-a414-444b-b837-26730da69797 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:24.905541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:24.966777 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede281b-b93f-4874-8894-0fe28997edcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:24.967421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede281b-b93f-4874-8894-0fe28997edcc service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:24.967699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede281b-b93f-4874-8894-0fe28997edcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:24.967971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede281b-b93f-4874-8894-0fe28997edcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:24.969596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bede281b-b93f-4874-8894-0fe28997edcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:25.076161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede281b-b93f-4874-8894-0fe28997edcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:25.077592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede281b-b93f-4874-8894-0fe28997edcc service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:25.096280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 728/1446] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:27.503005 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab258a3b-8fb8-431b-9d6f-e93f0a7d743e None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:27.503529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab258a3b-8fb8-431b-9d6f-e93f0a7d743e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:27.503529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab258a3b-8fb8-431b-9d6f-e93f0a7d743e None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:27.504847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:27.510130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f93c0-b487-4c34-8b8b-28431455da82 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:27.510581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f93c0-b487-4c34-8b8b-28431455da82 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:27.510692 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f93c0-b487-4c34-8b8b-28431455da82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:27.559535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:27.577892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1f8f93c0-b487-4c34-8b8b-28431455da82 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:27.579136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1f8f93c0-b487-4c34-8b8b-28431455da82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:27.678293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:27.716984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab258a3b-8fb8-431b-9d6f-e93f0a7d743e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:27.718537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ab258a3b-8fb8-431b-9d6f-e93f0a7d743e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:27.821071 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab258a3b-8fb8-431b-9d6f-e93f0a7d743e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:27.859851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:27.893336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5727c-ee6b-4766-8d09-697d8ba2f313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:27.894002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5727c-ee6b-4766-8d09-697d8ba2f313 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:27.894285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5727c-ee6b-4766-8d09-697d8ba2f313 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:27.894559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5727c-ee6b-4766-8d09-697d8ba2f313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:27.896185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-43c5727c-ee6b-4766-8d09-697d8ba2f313 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:28.001997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5727c-ee6b-4766-8d09-697d8ba2f313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:28.003422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5727c-ee6b-4766-8d09-697d8ba2f313 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:28.020720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 720/1449] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:30.117410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438f966-3682-4b47-839f-0e4f5786c9ea None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:30.117965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438f966-3682-4b47-839f-0e4f5786c9ea None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:30.118088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438f966-3682-4b47-839f-0e4f5786c9ea None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:30.119462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:30.124718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb278907-47ef-408b-9d58-b6d53aee5d7a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:30.125197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb278907-47ef-408b-9d58-b6d53aee5d7a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:30.125293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb278907-47ef-408b-9d58-b6d53aee5d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:30.190705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-bb278907-47ef-408b-9d58-b6d53aee5d7a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:30.191896 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-bb278907-47ef-408b-9d58-b6d53aee5d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:30.212312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:30.287711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:30.326147 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438f966-3682-4b47-839f-0e4f5786c9ea None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:30.327688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1438f966-3682-4b47-839f-0e4f5786c9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:30.434336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438f966-3682-4b47-839f-0e4f5786c9ea None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:30.454208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:30.507996 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943abc67-6611-4bef-a22c-3b36681d0d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:30.508649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943abc67-6611-4bef-a22c-3b36681d0d10 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:30.508934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943abc67-6611-4bef-a22c-3b36681d0d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:30.509210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943abc67-6611-4bef-a22c-3b36681d0d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:30.510894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-943abc67-6611-4bef-a22c-3b36681d0d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:30.618289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943abc67-6611-4bef-a22c-3b36681d0d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:30.619733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943abc67-6611-4bef-a22c-3b36681d0d10 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:30.638985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 731/1452] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:32.769731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242085-7d21-4bdd-a7a4-81e07b11bb6d None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:32.770380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242085-7d21-4bdd-a7a4-81e07b11bb6d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:32.770380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242085-7d21-4bdd-a7a4-81e07b11bb6d None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:32.771695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:32.776940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54664b8d-b0a7-4af8-b02f-fff138400c5d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:32.777349 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54664b8d-b0a7-4af8-b02f-fff138400c5d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:32.777494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54664b8d-b0a7-4af8-b02f-fff138400c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:32.847665 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:32.849994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-54664b8d-b0a7-4af8-b02f-fff138400c5d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:32.851757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-54664b8d-b0a7-4af8-b02f-fff138400c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:32.951289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:32.990647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242085-7d21-4bdd-a7a4-81e07b11bb6d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:32.992164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ea242085-7d21-4bdd-a7a4-81e07b11bb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:33.095540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242085-7d21-4bdd-a7a4-81e07b11bb6d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:33.143272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:33.167945 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635d047-519c-4399-8e73-ecaec75d245d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:33.168598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635d047-519c-4399-8e73-ecaec75d245d service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:33.168875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635d047-519c-4399-8e73-ecaec75d245d service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:33.169147 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635d047-519c-4399-8e73-ecaec75d245d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:33.171515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4635d047-519c-4399-8e73-ecaec75d245d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:33.280688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635d047-519c-4399-8e73-ecaec75d245d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:33.282168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635d047-519c-4399-8e73-ecaec75d245d service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:33.299648 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 723/1455] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:35.331717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0dbef-902c-48b9-987b-00693b0ea527 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:35.332313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0dbef-902c-48b9-987b-00693b0ea527 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:35.332313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0dbef-902c-48b9-987b-00693b0ea527 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:35.333703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:35.338840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ceac-3838-41da-b2a1-56819b7c5c0f None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:35.339240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ceac-3838-41da-b2a1-56819b7c5c0f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:35.339392 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ceac-3838-41da-b2a1-56819b7c5c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:35.404470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-3591ceac-3838-41da-b2a1-56819b7c5c0f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:35.405798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-3591ceac-3838-41da-b2a1-56819b7c5c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:35.433248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:35.503228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:35.540913 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0dbef-902c-48b9-987b-00693b0ea527 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:35.542482 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-78b0dbef-902c-48b9-987b-00693b0ea527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:35.644145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0dbef-902c-48b9-987b-00693b0ea527 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:35.671663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:35.715729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258a451-24e7-4fcc-b3d8-8e4838461a8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:35.716412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258a451-24e7-4fcc-b3d8-8e4838461a8c service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:35.716693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258a451-24e7-4fcc-b3d8-8e4838461a8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:35.716968 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258a451-24e7-4fcc-b3d8-8e4838461a8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:35.718636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d258a451-24e7-4fcc-b3d8-8e4838461a8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:35.825280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258a451-24e7-4fcc-b3d8-8e4838461a8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:35.826760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258a451-24e7-4fcc-b3d8-8e4838461a8c service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:35.845074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 734/1458] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:37.876639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24851351-5869-42a8-b810-5f69578f969e None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:37.877205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24851351-5869-42a8-b810-5f69578f969e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:37.877205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24851351-5869-42a8-b810-5f69578f969e None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:37.878630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:37.883775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306d3b8-23af-441d-8c4f-944ec9713506 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:37.884201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306d3b8-23af-441d-8c4f-944ec9713506 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:37.884293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306d3b8-23af-441d-8c4f-944ec9713506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:37.949519 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2306d3b8-23af-441d-8c4f-944ec9713506 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:37.951270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2306d3b8-23af-441d-8c4f-944ec9713506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:37.959446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:38.058046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:38.095948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24851351-5869-42a8-b810-5f69578f969e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:38.097482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-24851351-5869-42a8-b810-5f69578f969e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:38.199183 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24851351-5869-42a8-b810-5f69578f969e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:38.254331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:38.273414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234230c-c21e-4c78-ac14-8cd2c274e522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:38.274092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234230c-c21e-4c78-ac14-8cd2c274e522 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:38.274374 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234230c-c21e-4c78-ac14-8cd2c274e522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:38.274643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234230c-c21e-4c78-ac14-8cd2c274e522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:38.276270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f234230c-c21e-4c78-ac14-8cd2c274e522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:38.385456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234230c-c21e-4c78-ac14-8cd2c274e522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:38.386913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234230c-c21e-4c78-ac14-8cd2c274e522 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:38.404843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 726/1461] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:40.412775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2742936-f697-458e-a9c4-1e44f40fbbd9 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:40.413349 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2742936-f697-458e-a9c4-1e44f40fbbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:40.413349 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2742936-f697-458e-a9c4-1e44f40fbbd9 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:40.414716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:40.420547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0625414-108d-4afc-9c7a-d7aac9bb727c None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:40.420968 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0625414-108d-4afc-9c7a-d7aac9bb727c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:40.421193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0625414-108d-4afc-9c7a-d7aac9bb727c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:40.486720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-e0625414-108d-4afc-9c7a-d7aac9bb727c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:40.487854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e0625414-108d-4afc-9c7a-d7aac9bb727c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:40.525246 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:40.582628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:33:40.620719 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2742936-f697-458e-a9c4-1e44f40fbbd9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:40.622295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a2742936-f697-458e-a9c4-1e44f40fbbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:40.723549 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2742936-f697-458e-a9c4-1e44f40fbbd9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:40.759809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:40.795974 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6f7e1-9445-4b66-9359-d57f9092fcc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:40.796627 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6f7e1-9445-4b66-9359-d57f9092fcc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:40.796910 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6f7e1-9445-4b66-9359-d57f9092fcc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:40.797187 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6f7e1-9445-4b66-9359-d57f9092fcc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:40.798940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fea6f7e1-9445-4b66-9359-d57f9092fcc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:40.906297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6f7e1-9445-4b66-9359-d57f9092fcc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:40.907773 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6f7e1-9445-4b66-9359-d57f9092fcc9 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:40.925928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 737/1464] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:42.934643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9456146-6127-429a-9e54-977e80ed811a None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:42.935151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9456146-6127-429a-9e54-977e80ed811a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:42.935257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9456146-6127-429a-9e54-977e80ed811a None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:42.936645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:42.942031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70f43-def8-4bbf-94ad-5a08d1aa2260 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:42.942503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70f43-def8-4bbf-94ad-5a08d1aa2260 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:42.942972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70f43-def8-4bbf-94ad-5a08d1aa2260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:43.012496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-64f70f43-def8-4bbf-94ad-5a08d1aa2260 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:43.014260 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-64f70f43-def8-4bbf-94ad-5a08d1aa2260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:43.032411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:43.110919 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:43.147453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9456146-6127-429a-9e54-977e80ed811a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:43.148970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c9456146-6127-429a-9e54-977e80ed811a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:43.249375 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9456146-6127-429a-9e54-977e80ed811a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:43.314115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:43.320994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc153a-73a8-4b41-b828-1334602781e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:43.321639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc153a-73a8-4b41-b828-1334602781e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:43.321936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc153a-73a8-4b41-b828-1334602781e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:43.322337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc153a-73a8-4b41-b828-1334602781e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:43.323867 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-82fc153a-73a8-4b41-b828-1334602781e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:43.431516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc153a-73a8-4b41-b828-1334602781e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:43.432869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc153a-73a8-4b41-b828-1334602781e3 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:43.451691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 729/1467] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:45.437458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b97c7f-08cf-43fb-a0cf-70c74fec4317 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:45.437993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b97c7f-08cf-43fb-a0cf-70c74fec4317 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:45.437993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b97c7f-08cf-43fb-a0cf-70c74fec4317 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:45.439363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:33:45.444657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086aac3b-22d1-4fd7-9efa-f3156ca065d4 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:45.445048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086aac3b-22d1-4fd7-9efa-f3156ca065d4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:45.445214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086aac3b-22d1-4fd7-9efa-f3156ca065d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:45.512450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-086aac3b-22d1-4fd7-9efa-f3156ca065d4 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:33:45.514187 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-086aac3b-22d1-4fd7-9efa-f3156ca065d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:45.679129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:45.729398 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:33:45.766002 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b97c7f-08cf-43fb-a0cf-70c74fec4317 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:33:45.767540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a3b97c7f-08cf-43fb-a0cf-70c74fec4317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:45.869239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b97c7f-08cf-43fb-a0cf-70c74fec4317 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:33:45.915904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:45.941707 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1e6b0-cf7e-47eb-942e-f56d10d39699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:33:45.942392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1e6b0-cf7e-47eb-942e-f56d10d39699 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:33:45.942674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1e6b0-cf7e-47eb-942e-f56d10d39699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:33:45.942978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1e6b0-cf7e-47eb-942e-f56d10d39699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:33:45.944627 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-48c1e6b0-cf7e-47eb-942e-f56d10d39699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:33:46.050442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1e6b0-cf7e-47eb-942e-f56d10d39699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:33:46.051872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1e6b0-cf7e-47eb-942e-f56d10d39699 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:33:46.070165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 740/1470] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:33:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:55.727441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=22 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:33:56.067868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=66 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:05.561878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792af6a-3a78-4887-98da-74cc61e36057 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:05.562476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792af6a-3a78-4887-98da-74cc61e36057 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:05.562476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792af6a-3a78-4887-98da-74cc61e36057 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:05.563824 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 731/1471] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:34:05.569155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566517e-d539-426c-9865-04a4f2510e8b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:05.569579 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566517e-d539-426c-9865-04a4f2510e8b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:05.569708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566517e-d539-426c-9865-04a4f2510e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:05.634934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c566517e-d539-426c-9865-04a4f2510e8b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:34:05.636143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c566517e-d539-426c-9865-04a4f2510e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:05.729934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 741/1472] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:34:05.813307 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682deaa-ce31-4570-a6f2-2b29c906736d None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:05.813598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682deaa-ce31-4570-a6f2-2b29c906736d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:05.813879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682deaa-ce31-4570-a6f2-2b29c906736d None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:05.815281 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 732/1473] 10.222.0.60 () {54 vars in 966 bytes} [Mon Jul 25 12:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:34:05.820177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df9d9b-9ca7-4f32-9e89-9f9fbc3b318a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:05.820466 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df9d9b-9ca7-4f32-9e89-9f9fbc3b318a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:05.820746 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df9d9b-9ca7-4f32-9e89-9f9fbc3b318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:05.897509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-05df9d9b-9ca7-4f32-9e89-9f9fbc3b318a None None] MFA Rules not processed for user `6b07dcd8c0de4d1694d4363520726df2`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:34:05.898739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-05df9d9b-9ca7-4f32-9e89-9f9fbc3b318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:05.918789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:05.974297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 742/1474] 10.222.0.60 () {58 vars in 1063 bytes} [Mon Jul 25 12:34:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:34:05.982166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682deaa-ce31-4570-a6f2-2b29c906736d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:34:05.983584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d682deaa-ce31-4570-a6f2-2b29c906736d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:06.084251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682deaa-ce31-4570-a6f2-2b29c906736d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:34:06.156707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443adb06-96d7-4131-9e7b-9fffe8499fce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b07dcd8c0de4d1694d4363520726df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:34:06.157346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443adb06-96d7-4131-9e7b-9fffe8499fce service glance] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:06.157622 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443adb06-96d7-4131-9e7b-9fffe8499fce service glance] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:06.157891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443adb06-96d7-4131-9e7b-9fffe8499fce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:06.159530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-443adb06-96d7-4131-9e7b-9fffe8499fce service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:06.211806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:06.265952 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443adb06-96d7-4131-9e7b-9fffe8499fce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:34:06.267381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443adb06-96d7-4131-9e7b-9fffe8499fce service glance] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:34:06.285337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 733/1475] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:34:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:34:08.345735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba129514-9e9b-4183-bacb-abbfc8f8b6ff None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:08.346319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba129514-9e9b-4183-bacb-abbfc8f8b6ff None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:08.346447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba129514-9e9b-4183-bacb-abbfc8f8b6ff None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:08.347829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 743/1476] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:34:08.353265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd140f0a-74d8-46c8-ac48-44402b931227 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:08.353720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd140f0a-74d8-46c8-ac48-44402b931227 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:08.353839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd140f0a-74d8-46c8-ac48-44402b931227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:08.417311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-bd140f0a-74d8-46c8-ac48-44402b931227 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:34:08.418518 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-bd140f0a-74d8-46c8-ac48-44402b931227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:08.512961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 734/1477] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:34:08.535651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba129514-9e9b-4183-bacb-abbfc8f8b6ff None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:34:08.537216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ba129514-9e9b-4183-bacb-abbfc8f8b6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:08.662483 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba129514-9e9b-4183-bacb-abbfc8f8b6ff None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:34:08.710172 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:08.755293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5530f0-26f0-4f64-a323-5a93fcc4fcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:34:08.755951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5530f0-26f0-4f64-a323-5a93fcc4fcdb admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:08.756228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5530f0-26f0-4f64-a323-5a93fcc4fcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:08.756497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5530f0-26f0-4f64-a323-5a93fcc4fcdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:08.757218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5530f0-26f0-4f64-a323-5a93fcc4fcdb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:34:08.758958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-4c5530f0-26f0-4f64-a323-5a93fcc4fcdb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:34:08.758958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 25 12:34:08.758958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: Jul 25 12:34:08.760104 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5530f0-26f0-4f64-a323-5a93fcc4fcdb admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:34:08.778393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 744/1478] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Jul 25 12:34:08 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:34:10.794839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df6a60-e5a0-4d54-9221-646fb5fe1d3a None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:10.795401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df6a60-e5a0-4d54-9221-646fb5fe1d3a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:10.795401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df6a60-e5a0-4d54-9221-646fb5fe1d3a None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:10.796769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 735/1479] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:34:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:34:10.802010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9a4fa-5f38-4720-ad01-120f3712aa49 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:10.802432 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9a4fa-5f38-4720-ad01-120f3712aa49 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:10.802592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9a4fa-5f38-4720-ad01-120f3712aa49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:10.867348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-71f9a4fa-5f38-4720-ad01-120f3712aa49 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:34:10.868546 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-71f9a4fa-5f38-4720-ad01-120f3712aa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:10.964836 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 745/1480] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:34:11.007824 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df6a60-e5a0-4d54-9221-646fb5fe1d3a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:34:11.009400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d1df6a60-e5a0-4d54-9221-646fb5fe1d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:11.018865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:11.153476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df6a60-e5a0-4d54-9221-646fb5fe1d3a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:34:11.233791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:11.266089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62268fa-31fc-4c59-8917-aa9195a8765e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:34:11.266737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62268fa-31fc-4c59-8917-aa9195a8765e service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:11.267014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62268fa-31fc-4c59-8917-aa9195a8765e service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:11.267285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62268fa-31fc-4c59-8917-aa9195a8765e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:11.268892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b62268fa-31fc-4c59-8917-aa9195a8765e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:11.356033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62268fa-31fc-4c59-8917-aa9195a8765e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:34:11.358249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62268fa-31fc-4c59-8917-aa9195a8765e service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:34:11.376745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 736/1481] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:34:13.558769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f447c1-c739-4ebb-81b7-4f9ab630bb2e None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:13.559287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f447c1-c739-4ebb-81b7-4f9ab630bb2e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:13.559409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f447c1-c739-4ebb-81b7-4f9ab630bb2e None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:13.560739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 746/1482] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:34:13.567162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a36e23-a99b-4c5a-95ed-b96dfa0ee27d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:13.567598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a36e23-a99b-4c5a-95ed-b96dfa0ee27d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:13.567691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a36e23-a99b-4c5a-95ed-b96dfa0ee27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:13.632209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-36a36e23-a99b-4c5a-95ed-b96dfa0ee27d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:34:13.633400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-36a36e23-a99b-4c5a-95ed-b96dfa0ee27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:13.678534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:13.730126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 737/1483] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:34:13.767536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f447c1-c739-4ebb-81b7-4f9ab630bb2e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:34:13.769071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-29f447c1-c739-4ebb-81b7-4f9ab630bb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:13.793103 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:13.872536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f447c1-c739-4ebb-81b7-4f9ab630bb2e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:34:13.943766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a4d2a-7956-4dfe-aed7-d0f188525714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:34:13.944411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a4d2a-7956-4dfe-aed7-d0f188525714 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:13.944695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a4d2a-7956-4dfe-aed7-d0f188525714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:13.945083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a4d2a-7956-4dfe-aed7-d0f188525714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:13.946769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-dd0a4d2a-7956-4dfe-aed7-d0f188525714 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:14.019553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:14.054359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a4d2a-7956-4dfe-aed7-d0f188525714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:34:14.055789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a4d2a-7956-4dfe-aed7-d0f188525714 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:34:14.073765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 747/1484] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:34:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:34:16.127688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c0a17-02f6-483e-8bf6-c1066d84c910 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:16.128277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c0a17-02f6-483e-8bf6-c1066d84c910 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:16.128277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c0a17-02f6-483e-8bf6-c1066d84c910 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:16.129612 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 738/1485] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:34:16.134977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07163a95-1a66-454b-9168-f9a61fed2fd3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:16.135397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07163a95-1a66-454b-9168-f9a61fed2fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:16.135505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07163a95-1a66-454b-9168-f9a61fed2fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:16.200512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-07163a95-1a66-454b-9168-f9a61fed2fd3 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:34:16.201723 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-07163a95-1a66-454b-9168-f9a61fed2fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:16.297188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 748/1486] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:34:16.335930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c0a17-02f6-483e-8bf6-c1066d84c910 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:34:16.337505 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7c1c0a17-02f6-483e-8bf6-c1066d84c910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:16.437377 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c0a17-02f6-483e-8bf6-c1066d84c910 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:34:16.505890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:16.507676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412df51c-b385-44bd-8ed2-38bbf3040452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:34:16.508324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412df51c-b385-44bd-8ed2-38bbf3040452 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:16.508867 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412df51c-b385-44bd-8ed2-38bbf3040452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:16.509212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412df51c-b385-44bd-8ed2-38bbf3040452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:16.511311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-412df51c-b385-44bd-8ed2-38bbf3040452 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:16.622092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412df51c-b385-44bd-8ed2-38bbf3040452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:34:16.623559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412df51c-b385-44bd-8ed2-38bbf3040452 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:34:16.641179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 739/1487] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:34:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:34:18.814943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b4745-a9b1-4ee6-a73d-8e1566893aad None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:18.815506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b4745-a9b1-4ee6-a73d-8e1566893aad None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:18.815506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b4745-a9b1-4ee6-a73d-8e1566893aad None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:18.816858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 749/1488] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:34:18.822000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5fee1-a79a-4de7-bf53-d26e7c51be83 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:18.822446 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5fee1-a79a-4de7-bf53-d26e7c51be83 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:18.822554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5fee1-a79a-4de7-bf53-d26e7c51be83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:18.887781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-07e5fee1-a79a-4de7-bf53-d26e7c51be83 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:34:18.889595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-07e5fee1-a79a-4de7-bf53-d26e7c51be83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:18.941329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:18.986396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 740/1489] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:34:19.027004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b4745-a9b1-4ee6-a73d-8e1566893aad None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:34:19.028514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bf7b4745-a9b1-4ee6-a73d-8e1566893aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:19.065270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:19.139527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b4745-a9b1-4ee6-a73d-8e1566893aad None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:34:19.217401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073ee4b-39a7-4d06-b9fe-ab70ac18981f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:34:19.218251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073ee4b-39a7-4d06-b9fe-ab70ac18981f service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:19.218533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073ee4b-39a7-4d06-b9fe-ab70ac18981f service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:19.218808 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073ee4b-39a7-4d06-b9fe-ab70ac18981f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:19.220431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9073ee4b-39a7-4d06-b9fe-ab70ac18981f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:19.298383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:19.327636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073ee4b-39a7-4d06-b9fe-ab70ac18981f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:34:19.329008 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073ee4b-39a7-4d06-b9fe-ab70ac18981f service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:34:19.347787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 750/1490] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:34:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:34:21.664170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abb449-8e4c-4702-8daa-0356e588bdd1 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:21.664724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abb449-8e4c-4702-8daa-0356e588bdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:21.664724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abb449-8e4c-4702-8daa-0356e588bdd1 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:21.666119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 741/1491] 10.222.0.60 () {54 vars in 926 bytes} [Mon Jul 25 12:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:34:21.671385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e73ab-81f5-4f6b-b275-86239125221b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:21.671815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e73ab-81f5-4f6b-b275-86239125221b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:21.671943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e73ab-81f5-4f6b-b275-86239125221b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:21.736441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3b3e73ab-81f5-4f6b-b275-86239125221b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:34:21.737634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3b3e73ab-81f5-4f6b-b275-86239125221b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:21.832202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 751/1492] 10.222.0.60 () {58 vars in 1023 bytes} [Mon Jul 25 12:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:34:21.871176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abb449-8e4c-4702-8daa-0356e588bdd1 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:34:21.873227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-71abb449-8e4c-4702-8daa-0356e588bdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:21.976160 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abb449-8e4c-4702-8daa-0356e588bdd1 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:34:22.047335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fda8f-ae57-4966-8773-4475130d0d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:34:22.047980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fda8f-ae57-4966-8773-4475130d0d4c service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:34:22.048260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fda8f-ae57-4966-8773-4475130d0d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:34:22.048534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fda8f-ae57-4966-8773-4475130d0d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:34:22.050199 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-cc1fda8f-ae57-4966-8773-4475130d0d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:34:22.059200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:22.162823 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fda8f-ae57-4966-8773-4475130d0d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:34:22.164249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fda8f-ae57-4966-8773-4475130d0d4c service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:34:22.182577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 742/1493] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:34:31.830457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=82 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:34:32.180270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=50 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:35:48.988635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4fddfe-825b-4402-8df7-1057b43d9470 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:48.989198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4fddfe-825b-4402-8df7-1057b43d9470 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:48.989198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4fddfe-825b-4402-8df7-1057b43d9470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:49.064455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-da4fddfe-825b-4402-8df7-1057b43d9470 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:35:49.065654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-da4fddfe-825b-4402-8df7-1057b43d9470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:49.159359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 752/1494] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:35:49.167578 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff670b8b-a603-471c-8768-508209809f1a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:49.168068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff670b8b-a603-471c-8768-508209809f1a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:49.168168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff670b8b-a603-471c-8768-508209809f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:49.243842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-ff670b8b-a603-471c-8768-508209809f1a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:35:49.244990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ff670b8b-a603-471c-8768-508209809f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:49.338334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 743/1495] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:35:49.343470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4fddfe-825b-4402-8df7-1057b43d9470 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:35:49.344652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-da4fddfe-825b-4402-8df7-1057b43d9470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:49.356066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4fddfe-825b-4402-8df7-1057b43d9470 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:35:49.415058 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:35:49.448553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f8d61-cc28-49e6-9f97-ebf173e4330e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:35:49.449210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f8d61-cc28-49e6-9f97-ebf173e4330e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:49.449487 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f8d61-cc28-49e6-9f97-ebf173e4330e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:49.449759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f8d61-cc28-49e6-9f97-ebf173e4330e admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:49.450559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f8d61-cc28-49e6-9f97-ebf173e4330e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:35:49.450861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f8d61-cc28-49e6-9f97-ebf173e4330e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:35:49.451949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-d30f8d61-cc28-49e6-9f97-ebf173e4330e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:35:49.451949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 25 12:35:49.451949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: Jul 25 12:35:49.452834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f8d61-cc28-49e6-9f97-ebf173e4330e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:35:49.463667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 753/1496] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:35:49.472259 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff670b8b-a603-471c-8768-508209809f1a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:35:49.481487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff670b8b-a603-471c-8768-508209809f1a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:35:49.540704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:35:49.573779 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be8b98-e9ac-40ea-91a5-d7ab4a958c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:35:49.574468 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be8b98-e9ac-40ea-91a5-d7ab4a958c5f admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:49.574749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be8b98-e9ac-40ea-91a5-d7ab4a958c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:49.575023 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be8b98-e9ac-40ea-91a5-d7ab4a958c5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:49.575748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be8b98-e9ac-40ea-91a5-d7ab4a958c5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:35:49.577582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: WARNING py.warnings [None req-24be8b98-e9ac-40ea-91a5-d7ab4a958c5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:35:49.577582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: warnings.warn(msg) Jul 25 12:35:49.577582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: Jul 25 12:35:49.579663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be8b98-e9ac-40ea-91a5-d7ab4a958c5f admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:35:49.637512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 744/1497] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:35:49 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:35:49.641892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f8d61-cc28-49e6-9f97-ebf173e4330e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:35:49.643454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d30f8d61-cc28-49e6-9f97-ebf173e4330e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:49.676802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f8d61-cc28-49e6-9f97-ebf173e4330e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:35:49.768261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63173a5d-0008-4746-8920-b635c44ede36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:35:49.768916 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63173a5d-0008-4746-8920-b635c44ede36 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:49.769195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63173a5d-0008-4746-8920-b635c44ede36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:49.769461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63173a5d-0008-4746-8920-b635c44ede36 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:49.770174 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63173a5d-0008-4746-8920-b635c44ede36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:35:49.771172 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-63173a5d-0008-4746-8920-b635c44ede36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:35:49.771172 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 25 12:35:49.771172 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: Jul 25 12:35:49.772370 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63173a5d-0008-4746-8920-b635c44ede36 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:35:49.814283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 754/1498] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:35:49 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:35:49.818379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be8b98-e9ac-40ea-91a5-d7ab4a958c5f admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:35:49.819928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-24be8b98-e9ac-40ea-91a5-d7ab4a958c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:49.886238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be8b98-e9ac-40ea-91a5-d7ab4a958c5f admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:35:49.967189 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:35:49.979323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c79a62-739b-458a-abb1-93e0dbd02893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:35:49.980750 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c79a62-739b-458a-abb1-93e0dbd02893 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:49.981187 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c79a62-739b-458a-abb1-93e0dbd02893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:49.981505 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c79a62-739b-458a-abb1-93e0dbd02893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:49.982961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c79a62-739b-458a-abb1-93e0dbd02893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:35:49.984375 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: WARNING py.warnings [None req-99c79a62-739b-458a-abb1-93e0dbd02893 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:35:49.984375 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: warnings.warn(msg) Jul 25 12:35:49.984375 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: Jul 25 12:35:49.986105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c79a62-739b-458a-abb1-93e0dbd02893 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:35:49.998955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 745/1499] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:35:50.002955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63173a5d-0008-4746-8920-b635c44ede36 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:35:50.004467 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-63173a5d-0008-4746-8920-b635c44ede36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:50.016310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63173a5d-0008-4746-8920-b635c44ede36 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:35:50.040993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:35:50.110134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f9886-39bd-4834-bed5-32bc0f8702ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:35:50.110723 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f9886-39bd-4834-bed5-32bc0f8702ce admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:50.110999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f9886-39bd-4834-bed5-32bc0f8702ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:50.111269 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f9886-39bd-4834-bed5-32bc0f8702ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:50.111985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f9886-39bd-4834-bed5-32bc0f8702ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:35:50.112986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-001f9886-39bd-4834-bed5-32bc0f8702ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:35:50.112986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 25 12:35:50.112986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: Jul 25 12:35:50.113874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f9886-39bd-4834-bed5-32bc0f8702ce admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:35:50.126907 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 755/1500] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:35:50.130917 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c79a62-739b-458a-abb1-93e0dbd02893 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:35:50.132987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-99c79a62-739b-458a-abb1-93e0dbd02893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:50.167595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c79a62-739b-458a-abb1-93e0dbd02893 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:35:50.261412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c60ad-bda2-40e5-91c1-5e810178a669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:35:50.262330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c60ad-bda2-40e5-91c1-5e810178a669 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:50.262605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c60ad-bda2-40e5-91c1-5e810178a669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:50.262873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c60ad-bda2-40e5-91c1-5e810178a669 admin admin] PATH_INFO: `/v3/projects/01a366e92aaf4a02964359e72de1c551/users/1cafdcf18f5a455283b98800dfe419be/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:50.284668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c60ad-bda2-40e5-91c1-5e810178a669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a366e92aaf4a02964359e72de1c551, user_id=1cafdcf18f5a455283b98800dfe419be, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:35:50.285694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: WARNING py.warnings [None req-9a3c60ad-bda2-40e5-91c1-5e810178a669 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:35:50.285694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: warnings.warn(msg) Jul 25 12:35:50.285694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: Jul 25 12:35:50.287657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c60ad-bda2-40e5-91c1-5e810178a669 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:35:50.311722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 746/1501] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:35:50 2022] PUT /identity/v3/projects/01a366e92aaf4a02964359e72de1c551/users/1cafdcf18f5a455283b98800dfe419be/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:35:50.324045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad8068-69a3-4a78-a317-b481f3cdc724 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:50.324335 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad8068-69a3-4a78-a317-b481f3cdc724 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:50.324614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad8068-69a3-4a78-a317-b481f3cdc724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:50.381571 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-78ad8068-69a3-4a78-a317-b481f3cdc724 None None] MFA Rules not processed for user `1cafdcf18f5a455283b98800dfe419be`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:35:50.382755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-78ad8068-69a3-4a78-a317-b481f3cdc724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:50.458643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 756/1502] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:35:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:35:50.471843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f6e7b-d512-4917-84c8-32315dde7ab1 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:50.472167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f6e7b-d512-4917-84c8-32315dde7ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:50.472449 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f6e7b-d512-4917-84c8-32315dde7ab1 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:50.473851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 747/1503] 10.222.0.60 () {54 vars in 964 bytes} [Mon Jul 25 12:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:35:50.478862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd24b3-d927-48f0-9d67-40e6a3aa8666 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:50.479152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd24b3-d927-48f0-9d67-40e6a3aa8666 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:50.479428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd24b3-d927-48f0-9d67-40e6a3aa8666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:50.511009 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:35:50.554224 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e2cd24b3-d927-48f0-9d67-40e6a3aa8666 None None] MFA Rules not processed for user `eb4c5027951c4c55bd826943cec15309`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:35:50.555413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e2cd24b3-d927-48f0-9d67-40e6a3aa8666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:50.630831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 757/1504] 10.222.0.60 () {58 vars in 1061 bytes} [Mon Jul 25 12:35:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:35:50.638432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f6e7b-d512-4917-84c8-32315dde7ab1 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:35:50.639881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ee4f6e7b-d512-4917-84c8-32315dde7ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:50.689241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:35:50.742704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f6e7b-d512-4917-84c8-32315dde7ab1 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:35:50.812988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1307126-e395-4c34-9594-7e7e77afe103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:35:50.813624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1307126-e395-4c34-9594-7e7e77afe103 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:50.813902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1307126-e395-4c34-9594-7e7e77afe103 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:50.814201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1307126-e395-4c34-9594-7e7e77afe103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:50.815814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e1307126-e395-4c34-9594-7e7e77afe103 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:50.912922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1307126-e395-4c34-9594-7e7e77afe103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:35:50.914303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1307126-e395-4c34-9594-7e7e77afe103 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:35:50.917730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:35:50.932852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 748/1505] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:35:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:35:50.977446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a5f97-dda6-4fa3-9605-11f5a0b3e68e None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:50.977735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a5f97-dda6-4fa3-9605-11f5a0b3e68e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:50.978054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a5f97-dda6-4fa3-9605-11f5a0b3e68e None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:50.979361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 758/1506] 10.222.0.60 () {52 vars in 830 bytes} [Mon Jul 25 12:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:35:50.983783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1307126-e395-4c34-9594-7e7e77afe103 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:35:50.985014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e1307126-e395-4c34-9594-7e7e77afe103 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:51.082135 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1307126-e395-4c34-9594-7e7e77afe103 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:35:51.177357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:35:51.178188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:51.178471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:51.178747 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] PATH_INFO: `/v3/users/1cafdcf18f5a455283b98800dfe419be` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:51.191378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cafdcf18f5a455283b98800dfe419be)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:35:51.192405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: WARNING py.warnings [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:35:51.192405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: warnings.warn(msg) Jul 25 12:35:51.192405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: Jul 25 12:35:51.193619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:35:51.203608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:35:51.256461 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cafdcf18f5a455283b98800dfe419be'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:35:51.256894 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cafdcf18f5a455283b98800dfe419be'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:35:51.257440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cafdcf18f5a455283b98800dfe419be'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:35:51.269856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cafdcf18f5a455283b98800dfe419be'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:35:51.279773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cafdcf18f5a455283b98800dfe419be'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:35:51.304637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 749/1507] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:35:50 2022] DELETE /identity/v3/users/1cafdcf18f5a455283b98800dfe419be => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:35:51.309231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cfe73-67e8-4d00-9394-b26b67fb7579 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:51.309741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cfe73-67e8-4d00-9394-b26b67fb7579 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:51.309845 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cfe73-67e8-4d00-9394-b26b67fb7579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:51.367533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7f3cfe73-67e8-4d00-9394-b26b67fb7579 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:35:51.368731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7f3cfe73-67e8-4d00-9394-b26b67fb7579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:51.441399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:35:51.454517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 759/1508] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:35:51.464141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:35:51.465357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:51.567105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08309d83-0b57-40ea-97f4-d676222b5f46 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:35:51.620976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:35:51.643298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae89b4f-3e14-40e8-9d58-6e0c9539d5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:35:51.643943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae89b4f-3e14-40e8-9d58-6e0c9539d5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:51.644227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae89b4f-3e14-40e8-9d58-6e0c9539d5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:51.644505 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae89b4f-3e14-40e8-9d58-6e0c9539d5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:51.646176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-cae89b4f-3e14-40e8-9d58-6e0c9539d5a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:51.752769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae89b4f-3e14-40e8-9d58-6e0c9539d5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:35:51.754158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae89b4f-3e14-40e8-9d58-6e0c9539d5a3 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:35:51.772424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 750/1509] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:35:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:35:52.095220 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cfe73-67e8-4d00-9394-b26b67fb7579 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:35:52.096443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7f3cfe73-67e8-4d00-9394-b26b67fb7579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:35:52.108714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cfe73-67e8-4d00-9394-b26b67fb7579 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:35:52.205595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:35:52.206488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:35:52.206772 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:35:52.207048 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] PATH_INFO: `/v3/projects/01a366e92aaf4a02964359e72de1c551` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:35:52.220086 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=01a366e92aaf4a02964359e72de1c551)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:35:52.221270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:35:52.221270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 25 12:35:52.221270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: Jul 25 12:35:52.222647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:35:52.282796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01a366e92aaf4a02964359e72de1c551', 'request_id': 'req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:35:52.292175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01a366e92aaf4a02964359e72de1c551', 'request_id': 'req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:35:52.305620 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01a366e92aaf4a02964359e72de1c551', 'request_id': 'req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:35:52.329596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] The token cache is being invalidate because project 01a366e92aaf4a02964359e72de1c551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:35:52.330210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:35:52.330419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-67855b9b-4fe6-4f50-a62d-485e12c4b6bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:35:52.331785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 760/1510] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:35:52 2022] DELETE /identity/v3/projects/01a366e92aaf4a02964359e72de1c551 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:36:01.770154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=62 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:36:02.327645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:37:28.129024 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c88361-91aa-49ba-acdc-323c903d3579 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:28.129632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c88361-91aa-49ba-acdc-323c903d3579 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:28.129632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c88361-91aa-49ba-acdc-323c903d3579 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:28.131047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:37:28.135807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eb0c9-d867-4379-870e-44fe4a870ec0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:28.136218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eb0c9-d867-4379-870e-44fe4a870ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:28.136358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eb0c9-d867-4379-870e-44fe4a870ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:28.216815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-599eb0c9-d867-4379-870e-44fe4a870ec0 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:37:28.218574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-599eb0c9-d867-4379-870e-44fe4a870ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:28.320888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:37:28.477228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c88361-91aa-49ba-acdc-323c903d3579 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:37:28.479366 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d4c88361-91aa-49ba-acdc-323c903d3579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:28.597219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c88361-91aa-49ba-acdc-323c903d3579 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:37:28.602691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b8354-7c0e-4bf0-b277-b448e87320b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:37:28.603333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b8354-7c0e-4bf0-b277-b448e87320b6 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:28.603616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b8354-7c0e-4bf0-b277-b448e87320b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:28.603897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b8354-7c0e-4bf0-b277-b448e87320b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:28.612272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b8354-7c0e-4bf0-b277-b448e87320b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:37:28.613837 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b8354-7c0e-4bf0-b277-b448e87320b6 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:37:28.623072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 752/1513] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:37:29.447052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e4d7a-58b7-4cee-a37d-80a78db34a27 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:29.447641 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e4d7a-58b7-4cee-a37d-80a78db34a27 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:29.447641 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e4d7a-58b7-4cee-a37d-80a78db34a27 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:29.448969 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:37:29.453710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411937f5-fca5-4a92-aa7a-2f6946b9d038 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:29.454143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411937f5-fca5-4a92-aa7a-2f6946b9d038 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:29.454297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411937f5-fca5-4a92-aa7a-2f6946b9d038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:29.473765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-411937f5-fca5-4a92-aa7a-2f6946b9d038 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:37:29.474980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-411937f5-fca5-4a92-aa7a-2f6946b9d038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:29.482790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:37:29.633843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e4d7a-58b7-4cee-a37d-80a78db34a27 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:37:29.648083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e4d7a-58b7-4cee-a37d-80a78db34a27 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:37:29.653453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bf435-592e-4c74-9377-e98ca42f3479 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:37:29.654125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bf435-592e-4c74-9377-e98ca42f3479 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:29.654410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bf435-592e-4c74-9377-e98ca42f3479 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:29.654685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bf435-592e-4c74-9377-e98ca42f3479 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:29.663220 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bf435-592e-4c74-9377-e98ca42f3479 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:37:29.664800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bf435-592e-4c74-9377-e98ca42f3479 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:37:29.674720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 763/1516] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:37:30.179647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88e1f9-feff-4916-9e73-10fb8290a672 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:30.179930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88e1f9-feff-4916-9e73-10fb8290a672 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:30.180207 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88e1f9-feff-4916-9e73-10fb8290a672 None None] PATH_INFO: `/v3` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:30.181517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 754/1517] 10.222.0.61 () {54 vars in 925 bytes} [Mon Jul 25 12:37:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:37:30.187635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac3364-f6b6-4733-a5b0-ebddf0e9b31a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:30.187924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac3364-f6b6-4733-a5b0-ebddf0e9b31a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:30.188208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac3364-f6b6-4733-a5b0-ebddf0e9b31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:30.246053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ecac3364-f6b6-4733-a5b0-ebddf0e9b31a None None] MFA Rules not processed for user `81037f61e01e4a93982503720f7404e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:37:30.247244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ecac3364-f6b6-4733-a5b0-ebddf0e9b31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:30.263308 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:37:30.316358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 764/1518] 10.222.0.61 () {58 vars in 1015 bytes} [Mon Jul 25 12:37:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:37:39.121778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5020a-8407-4ad1-8eaf-43fe88680399 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:39.122362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5020a-8407-4ad1-8eaf-43fe88680399 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:39.122362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5020a-8407-4ad1-8eaf-43fe88680399 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:39.123653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 755/1519] 10.222.0.60 () {54 vars in 961 bytes} [Mon Jul 25 12:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:37:39.129011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f53257-0bab-43c8-a6b0-d31976262bec None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:39.129413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f53257-0bab-43c8-a6b0-d31976262bec None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:39.129564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f53257-0bab-43c8-a6b0-d31976262bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:39.148477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b2f53257-0bab-43c8-a6b0-d31976262bec None None] MFA Rules not processed for user `81037f61e01e4a93982503720f7404e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:37:39.149647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b2f53257-0bab-43c8-a6b0-d31976262bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:39.158962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 765/1520] 10.222.0.60 () {58 vars in 1058 bytes} [Mon Jul 25 12:37:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:37:39.166071 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5020a-8407-4ad1-8eaf-43fe88680399 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:37:39.180351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5020a-8407-4ad1-8eaf-43fe88680399 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:37:39.185331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e31e3-6e46-486d-a4d5-890b4f9a74ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:37:39.185997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e31e3-6e46-486d-a4d5-890b4f9a74ff service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:39.186283 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e31e3-6e46-486d-a4d5-890b4f9a74ff service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:39.186563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e31e3-6e46-486d-a4d5-890b4f9a74ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:39.194877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e31e3-6e46-486d-a4d5-890b4f9a74ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:37:39.196454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e31e3-6e46-486d-a4d5-890b4f9a74ff service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:37:39.205999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 756/1521] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:37:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:37:40.887540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91731351-61e2-41a7-89c2-cb834ec706ff None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:40.888114 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91731351-61e2-41a7-89c2-cb834ec706ff None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:40.888114 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91731351-61e2-41a7-89c2-cb834ec706ff None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:40.889503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:37:40.894900 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a89c3-c402-4b11-aee0-e4b6024cbb74 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:40.895337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a89c3-c402-4b11-aee0-e4b6024cbb74 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:40.895455 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a89c3-c402-4b11-aee0-e4b6024cbb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:40.914392 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-868a89c3-c402-4b11-aee0-e4b6024cbb74 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:37:40.915587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-868a89c3-c402-4b11-aee0-e4b6024cbb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:40.923652 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:37:41.077849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91731351-61e2-41a7-89c2-cb834ec706ff None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:37:41.092410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91731351-61e2-41a7-89c2-cb834ec706ff None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:37:41.097882 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7adfb8-c789-495d-8b42-36c17a962941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:37:41.098548 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7adfb8-c789-495d-8b42-36c17a962941 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:41.098834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7adfb8-c789-495d-8b42-36c17a962941 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:41.099115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7adfb8-c789-495d-8b42-36c17a962941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:41.107238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7adfb8-c789-495d-8b42-36c17a962941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:37:41.110347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7adfb8-c789-495d-8b42-36c17a962941 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:37:41.119810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 767/1524] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:37:41.949771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49af664-6f67-4937-900e-00b06696e01b None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:41.950375 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49af664-6f67-4937-900e-00b06696e01b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:41.950479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49af664-6f67-4937-900e-00b06696e01b None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:41.951879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:37:41.957240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86bd82-70d6-460f-8f04-372de04897a5 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:41.957718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86bd82-70d6-460f-8f04-372de04897a5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:41.957825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86bd82-70d6-460f-8f04-372de04897a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:41.977444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5c86bd82-70d6-460f-8f04-372de04897a5 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:37:41.978653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5c86bd82-70d6-460f-8f04-372de04897a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:41.987436 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:37:42.141814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49af664-6f67-4937-900e-00b06696e01b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:37:42.156596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49af664-6f67-4937-900e-00b06696e01b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:37:42.161963 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f1b58-6988-41e5-bcb8-d5671f10c089 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:37:42.162592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f1b58-6988-41e5-bcb8-d5671f10c089 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:42.162873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f1b58-6988-41e5-bcb8-d5671f10c089 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:42.163151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f1b58-6988-41e5-bcb8-d5671f10c089 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:42.171232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f1b58-6988-41e5-bcb8-d5671f10c089 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:37:42.172809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f1b58-6988-41e5-bcb8-d5671f10c089 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:37:42.182227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 759/1527] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:37:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:37:45.999881 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f61baa-afd7-4f83-8beb-08389cb2e4c7 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:46.000446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f61baa-afd7-4f83-8beb-08389cb2e4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:46.000446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f61baa-afd7-4f83-8beb-08389cb2e4c7 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:46.001826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:37:46.006946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1268584-afa5-4273-bc18-7738c6436f2f None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:46.007394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1268584-afa5-4273-bc18-7738c6436f2f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:46.007495 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1268584-afa5-4273-bc18-7738c6436f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:46.026479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b1268584-afa5-4273-bc18-7738c6436f2f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:37:46.027681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b1268584-afa5-4273-bc18-7738c6436f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:46.036163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:37:46.107379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f61baa-afd7-4f83-8beb-08389cb2e4c7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:37:46.109976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b9f61baa-afd7-4f83-8beb-08389cb2e4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:46.188287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f61baa-afd7-4f83-8beb-08389cb2e4c7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:37:46.191938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abae9d-793f-4740-8a53-5baa3e25c42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:37:46.192584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abae9d-793f-4740-8a53-5baa3e25c42f service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:46.192870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abae9d-793f-4740-8a53-5baa3e25c42f service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:46.193148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abae9d-793f-4740-8a53-5baa3e25c42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:46.202602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abae9d-793f-4740-8a53-5baa3e25c42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:37:46.204034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abae9d-793f-4740-8a53-5baa3e25c42f service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:37:46.213371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 770/1530] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:37:48.194090 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3690eb-5808-40f1-a44e-a8c7153537ca None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:48.194499 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3690eb-5808-40f1-a44e-a8c7153537ca None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:48.194643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3690eb-5808-40f1-a44e-a8c7153537ca None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:48.196024 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:37:48.201195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874a2b6-92b6-4593-ab04-e777e77b023a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:48.201640 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874a2b6-92b6-4593-ab04-e777e77b023a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:48.201745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874a2b6-92b6-4593-ab04-e777e77b023a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:48.221363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f874a2b6-92b6-4593-ab04-e777e77b023a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:37:48.222569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f874a2b6-92b6-4593-ab04-e777e77b023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:48.230264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:37:48.266478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3690eb-5808-40f1-a44e-a8c7153537ca None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:37:48.276936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3690eb-5808-40f1-a44e-a8c7153537ca None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:37:48.280482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de6a6c-db0f-47d1-a985-46f1ba6874a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:37:48.281114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de6a6c-db0f-47d1-a985-46f1ba6874a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:48.281396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de6a6c-db0f-47d1-a985-46f1ba6874a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:48.281670 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de6a6c-db0f-47d1-a985-46f1ba6874a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:48.290412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de6a6c-db0f-47d1-a985-46f1ba6874a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:37:48.291833 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de6a6c-db0f-47d1-a985-46f1ba6874a0 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:37:48.301068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 762/1533] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:37:51.751889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12920635-811c-4918-9cf6-c4ebfba7005d None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:51.752483 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12920635-811c-4918-9cf6-c4ebfba7005d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:51.752483 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12920635-811c-4918-9cf6-c4ebfba7005d None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:51.753800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:37:51.759063 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b81a76-482c-43fc-bd7c-6c1553e008ac None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:51.759505 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b81a76-482c-43fc-bd7c-6c1553e008ac None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:51.759613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b81a76-482c-43fc-bd7c-6c1553e008ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:51.778991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-a6b81a76-482c-43fc-bd7c-6c1553e008ac None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:37:51.780183 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a6b81a76-482c-43fc-bd7c-6c1553e008ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:51.787928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:37:52.089696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12920635-811c-4918-9cf6-c4ebfba7005d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:37:52.091375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-12920635-811c-4918-9cf6-c4ebfba7005d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:52.155033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:37:52.168662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12920635-811c-4918-9cf6-c4ebfba7005d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:37:52.172853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb48ca-733d-4ea6-8cea-8225085f1089 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:37:52.173504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb48ca-733d-4ea6-8cea-8225085f1089 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:52.173816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb48ca-733d-4ea6-8cea-8225085f1089 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:52.174135 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb48ca-733d-4ea6-8cea-8225085f1089 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:52.183680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb48ca-733d-4ea6-8cea-8225085f1089 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:37:52.185121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb48ca-733d-4ea6-8cea-8225085f1089 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:37:52.195138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 773/1536] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:37:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:37:54.182213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387b082-709b-42c9-b767-15fbe7d53b50 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:54.182771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387b082-709b-42c9-b767-15fbe7d53b50 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:54.182771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387b082-709b-42c9-b767-15fbe7d53b50 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:54.184133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:37:54.189522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3b8aa-3f56-4a41-a266-73e39701ce7b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:54.190013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3b8aa-3f56-4a41-a266-73e39701ce7b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:54.190180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3b8aa-3f56-4a41-a266-73e39701ce7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:54.210205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ddc3b8aa-3f56-4a41-a266-73e39701ce7b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:37:54.211443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ddc3b8aa-3f56-4a41-a266-73e39701ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:54.219937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:37:54.515419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387b082-709b-42c9-b767-15fbe7d53b50 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:37:54.517056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8387b082-709b-42c9-b767-15fbe7d53b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:37:54.524481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=84 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:37:54.531128 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387b082-709b-42c9-b767-15fbe7d53b50 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:37:54.534579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812d9b5-8344-43bd-946a-06ef0155c233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:37:54.535207 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812d9b5-8344-43bd-946a-06ef0155c233 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:37:54.535485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812d9b5-8344-43bd-946a-06ef0155c233 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:37:54.535758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812d9b5-8344-43bd-946a-06ef0155c233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:37:54.544391 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812d9b5-8344-43bd-946a-06ef0155c233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:37:54.545823 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812d9b5-8344-43bd-946a-06ef0155c233 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:37:54.555075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 765/1539] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:37:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:38:04.205380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=11 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:38:04.550930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:22.492228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a98b1-d21f-4ec4-a084-cead038c5c4c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:22.492912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a98b1-d21f-4ec4-a084-cead038c5c4c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:22.492912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a98b1-d21f-4ec4-a084-cead038c5c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:22.520745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f69a98b1-d21f-4ec4-a084-cead038c5c4c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:22.522056 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f69a98b1-d21f-4ec4-a084-cead038c5c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:22.537568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 775/1540] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:22.547217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c5a60-24ee-4d2f-9668-db8d9fcab3c6 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:22.548269 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c5a60-24ee-4d2f-9668-db8d9fcab3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:22.548782 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c5a60-24ee-4d2f-9668-db8d9fcab3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:22.595961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-e53c5a60-24ee-4d2f-9668-db8d9fcab3c6 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:22.597668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e53c5a60-24ee-4d2f-9668-db8d9fcab3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:22.611484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 766/1541] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:22.616222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a98b1-d21f-4ec4-a084-cead038c5c4c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:22.632417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a98b1-d21f-4ec4-a084-cead038c5c4c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:22.649774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d7a87-5d23-4789-8a6a-0bd7e0d7d262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:22.650566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d7a87-5d23-4789-8a6a-0bd7e0d7d262 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:22.650860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d7a87-5d23-4789-8a6a-0bd7e0d7d262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:22.651142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d7a87-5d23-4789-8a6a-0bd7e0d7d262 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:22.651893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d7a87-5d23-4789-8a6a-0bd7e0d7d262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:22.652203 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d7a87-5d23-4789-8a6a-0bd7e0d7d262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:22.653829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d7a87-5d23-4789-8a6a-0bd7e0d7d262 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:22.672010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 776/1542] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:39:22.681481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c5a60-24ee-4d2f-9668-db8d9fcab3c6 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:22.692790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c5a60-24ee-4d2f-9668-db8d9fcab3c6 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:22.702020 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51bba4a-b021-423c-8231-a6fabfc7b431 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:22.702377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51bba4a-b021-423c-8231-a6fabfc7b431 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:22.702630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51bba4a-b021-423c-8231-a6fabfc7b431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:22.703320 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8328db-8728-40ec-b321-cb5f5f0d7702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:22.704682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8328db-8728-40ec-b321-cb5f5f0d7702 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:22.705188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8328db-8728-40ec-b321-cb5f5f0d7702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:22.705797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8328db-8728-40ec-b321-cb5f5f0d7702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:22.706809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8328db-8728-40ec-b321-cb5f5f0d7702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:22.709055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8328db-8728-40ec-b321-cb5f5f0d7702 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:22.730725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c51bba4a-b021-423c-8231-a6fabfc7b431 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:22.731754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c51bba4a-b021-423c-8231-a6fabfc7b431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:22.754757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 777/1543] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:39:22.766177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b26172-c417-4b72-a39f-eab8b3e56b62 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:22.766485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b26172-c417-4b72-a39f-eab8b3e56b62 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:22.766745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b26172-c417-4b72-a39f-eab8b3e56b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:22.775381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 767/1544] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:22 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:22.783225 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8328db-8728-40ec-b321-cb5f5f0d7702 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:22.798063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-83b26172-c417-4b72-a39f-eab8b3e56b62 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:22.799287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-83b26172-c417-4b72-a39f-eab8b3e56b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:22.877986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 778/1545] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:39:22.879514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0a84e-0b75-44df-88e2-aed895d6acd1 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:22.879807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0a84e-0b75-44df-88e2-aed895d6acd1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:22.880088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0a84e-0b75-44df-88e2-aed895d6acd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:22.889513 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8328db-8728-40ec-b321-cb5f5f0d7702 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:22.900075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae73a7-3a87-497f-bad0-1416f7d9bbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:22.901029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9ee0a84e-0b75-44df-88e2-aed895d6acd1 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:22.901342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae73a7-3a87-497f-bad0-1416f7d9bbcb admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:22.901847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae73a7-3a87-497f-bad0-1416f7d9bbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:22.902280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9ee0a84e-0b75-44df-88e2-aed895d6acd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:22.902757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae73a7-3a87-497f-bad0-1416f7d9bbcb admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:22.903674 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae73a7-3a87-497f-bad0-1416f7d9bbcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:22.904915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: WARNING py.warnings [None req-8bae73a7-3a87-497f-bad0-1416f7d9bbcb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:39:22.904915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: warnings.warn(msg) Jul 25 12:39:22.904915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: Jul 25 12:39:22.906640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae73a7-3a87-497f-bad0-1416f7d9bbcb admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:22.912650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 779/1546] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:22.912650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0a84e-0b75-44df-88e2-aed895d6acd1 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:22.922627 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0a84e-0b75-44df-88e2-aed895d6acd1 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:22.930017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb564f9f-f2fd-430a-b14c-20d4fd814b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:22.930017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb564f9f-f2fd-430a-b14c-20d4fd814b2f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:22.930017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb564f9f-f2fd-430a-b14c-20d4fd814b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:22.930017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb564f9f-f2fd-430a-b14c-20d4fd814b2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:22.930404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb564f9f-f2fd-430a-b14c-20d4fd814b2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:22.930404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb564f9f-f2fd-430a-b14c-20d4fd814b2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:22.931699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb564f9f-f2fd-430a-b14c-20d4fd814b2f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:22.952243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 768/1547] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:22 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:39:22.953697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7696c-a4dc-4d69-9f1d-066daa81445a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:22.954007 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7696c-a4dc-4d69-9f1d-066daa81445a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:22.954289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7696c-a4dc-4d69-9f1d-066daa81445a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:22.957516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 780/1548] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:39:22.958542 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb564f9f-f2fd-430a-b14c-20d4fd814b2f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:22.982271 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb564f9f-f2fd-430a-b14c-20d4fd814b2f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:22.992256 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d369c-2a3d-4156-8d35-a38441a79235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:22.992946 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d369c-2a3d-4156-8d35-a38441a79235 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:22.993231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d369c-2a3d-4156-8d35-a38441a79235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:22.993507 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d369c-2a3d-4156-8d35-a38441a79235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:22.994287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d369c-2a3d-4156-8d35-a38441a79235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:22.995916 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d369c-2a3d-4156-8d35-a38441a79235 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.002121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-dbf7696c-a4dc-4d69-9f1d-066daa81445a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:23.003572 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-dbf7696c-a4dc-4d69-9f1d-066daa81445a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:23.021060 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 781/1549] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:23.023665 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 769/1550] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:23.024411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d369c-2a3d-4156-8d35-a38441a79235 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.026039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7696c-a4dc-4d69-9f1d-066daa81445a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.044204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d369c-2a3d-4156-8d35-a38441a79235 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.047069 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7696c-a4dc-4d69-9f1d-066daa81445a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.055801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28943496-dafa-48ac-a07c-5fa12c1ea98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.056494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28943496-dafa-48ac-a07c-5fa12c1ea98d admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.056781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28943496-dafa-48ac-a07c-5fa12c1ea98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.057060 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28943496-dafa-48ac-a07c-5fa12c1ea98d admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.057952 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28943496-dafa-48ac-a07c-5fa12c1ea98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.059453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28943496-dafa-48ac-a07c-5fa12c1ea98d admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.065938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75512e2-b0ff-453d-927c-10181a22dcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.065938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75512e2-b0ff-453d-927c-10181a22dcef admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.065938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75512e2-b0ff-453d-927c-10181a22dcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.065938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75512e2-b0ff-453d-927c-10181a22dcef admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.066380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75512e2-b0ff-453d-927c-10181a22dcef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.066380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-d75512e2-b0ff-453d-927c-10181a22dcef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:39:23.066380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 25 12:39:23.066380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: Jul 25 12:39:23.066380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75512e2-b0ff-453d-927c-10181a22dcef admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.091926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 770/1551] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:23.093627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28943496-dafa-48ac-a07c-5fa12c1ea98d admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.108151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28943496-dafa-48ac-a07c-5fa12c1ea98d admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.121173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 782/1552] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:23.122711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149b037-9301-43a6-b343-e19f07faabad None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.123018 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149b037-9301-43a6-b343-e19f07faabad None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.123303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149b037-9301-43a6-b343-e19f07faabad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.127070 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573377b-aea3-47d3-b210-766e4cc17e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.127763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573377b-aea3-47d3-b210-766e4cc17e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.128053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573377b-aea3-47d3-b210-766e4cc17e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.128336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573377b-aea3-47d3-b210-766e4cc17e31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.129078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573377b-aea3-47d3-b210-766e4cc17e31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:23.129404 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573377b-aea3-47d3-b210-766e4cc17e31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.130868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: WARNING py.warnings [None req-e573377b-aea3-47d3-b210-766e4cc17e31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:39:23.130868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: warnings.warn(msg) Jul 25 12:39:23.130868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: Jul 25 12:39:23.131759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573377b-aea3-47d3-b210-766e4cc17e31 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.152462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 771/1553] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:39:23.154071 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573377b-aea3-47d3-b210-766e4cc17e31 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.156448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3149b037-9301-43a6-b343-e19f07faabad None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:23.157660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3149b037-9301-43a6-b343-e19f07faabad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:23.169759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573377b-aea3-47d3-b210-766e4cc17e31 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.233022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 783/1554] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:23.235332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5bf0d-a7d1-48d4-b5c5-22fc91349d0b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.235720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5bf0d-a7d1-48d4-b5c5-22fc91349d0b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.236099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5bf0d-a7d1-48d4-b5c5-22fc91349d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.237961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cced7c2-14b7-4edc-861c-435d724f3a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.238976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cced7c2-14b7-4edc-861c-435d724f3a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.239317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cced7c2-14b7-4edc-861c-435d724f3a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.239533 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cced7c2-14b7-4edc-861c-435d724f3a34 admin admin] PATH_INFO: `/v3/projects/3b0ca10639aa47dea81db9799273acba/users/e56531920fc548a3a37f63ca0d1e5d82/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.254849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cced7c2-14b7-4edc-861c-435d724f3a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0ca10639aa47dea81db9799273acba, user_id=e56531920fc548a3a37f63ca0d1e5d82, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.255766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-21a5bf0d-a7d1-48d4-b5c5-22fc91349d0b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:23.256965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-21a5bf0d-a7d1-48d4-b5c5-22fc91349d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:23.257500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cced7c2-14b7-4edc-861c-435d724f3a34 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.266013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 784/1555] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:23.267511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504a344-c4c5-40fc-b8db-95e46fcf45f6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.267818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504a344-c4c5-40fc-b8db-95e46fcf45f6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.268277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504a344-c4c5-40fc-b8db-95e46fcf45f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.275598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 772/1556] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:23 2022] PUT /identity/v3/projects/3b0ca10639aa47dea81db9799273acba/users/e56531920fc548a3a37f63ca0d1e5d82/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:23.275598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cced7c2-14b7-4edc-861c-435d724f3a34 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.279371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:23.292905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7504a344-c4c5-40fc-b8db-95e46fcf45f6 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:23.294146 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7504a344-c4c5-40fc-b8db-95e46fcf45f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:23.303854 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:23.337858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cced7c2-14b7-4edc-861c-435d724f3a34 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.342848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3231c-9e12-409b-a624-8244c4fc841b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.343525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3231c-9e12-409b-a624-8244c4fc841b admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.343831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3231c-9e12-409b-a624-8244c4fc841b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.344135 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3231c-9e12-409b-a624-8244c4fc841b admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.344988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3231c-9e12-409b-a624-8244c4fc841b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.346945 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3231c-9e12-409b-a624-8244c4fc841b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.358651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 785/1557] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:23.359895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504a344-c4c5-40fc-b8db-95e46fcf45f6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.369964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504a344-c4c5-40fc-b8db-95e46fcf45f6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.375245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aa5c1-35b4-4416-9ffb-b25bf53d139e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.375896 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aa5c1-35b4-4416-9ffb-b25bf53d139e admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.376013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 773/1558] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:23.376176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aa5c1-35b4-4416-9ffb-b25bf53d139e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.376455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aa5c1-35b4-4416-9ffb-b25bf53d139e admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.377220 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2aa5c1-35b4-4416-9ffb-b25bf53d139e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.377682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12671094-069a-4006-8644-00a92c8dd91a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.378068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12671094-069a-4006-8644-00a92c8dd91a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.378411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12671094-069a-4006-8644-00a92c8dd91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.379647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2aa5c1-35b4-4416-9ffb-b25bf53d139e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.398444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-12671094-069a-4006-8644-00a92c8dd91a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:23.399876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-12671094-069a-4006-8644-00a92c8dd91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:23.408337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 774/1559] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:23.412015 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 786/1560] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:23.412178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e48f2e-b8f1-4bfd-a157-a7c05f8bc9c7 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.412178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e48f2e-b8f1-4bfd-a157-a7c05f8bc9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.412405 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4be60-fdd2-49eb-8bbf-e641dc02c866 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.412498 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e48f2e-b8f1-4bfd-a157-a7c05f8bc9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.412597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4be60-fdd2-49eb-8bbf-e641dc02c866 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.412597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4be60-fdd2-49eb-8bbf-e641dc02c866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.431149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-53e48f2e-b8f1-4bfd-a157-a7c05f8bc9c7 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:23.432364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-53e48f2e-b8f1-4bfd-a157-a7c05f8bc9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:23.432690 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-60c4be60-fdd2-49eb-8bbf-e641dc02c866 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:23.434000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-60c4be60-fdd2-49eb-8bbf-e641dc02c866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:23.493121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 775/1561] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:23.493655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 787/1562] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:23.494514 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b97cc8-1170-4625-96be-99f4dafb4c42 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.494806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b97cc8-1170-4625-96be-99f4dafb4c42 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.494919 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d841a-7552-4c05-a5f2-52c94efb25ca None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.495170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b97cc8-1170-4625-96be-99f4dafb4c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.495306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d841a-7552-4c05-a5f2-52c94efb25ca None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.495488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d841a-7552-4c05-a5f2-52c94efb25ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.515301 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-60b97cc8-1170-4625-96be-99f4dafb4c42 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:23.516508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-60b97cc8-1170-4625-96be-99f4dafb4c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:23.520674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-161d841a-7552-4c05-a5f2-52c94efb25ca None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:23.521844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-161d841a-7552-4c05-a5f2-52c94efb25ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:23.528237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 776/1563] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:23.528237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4113747-b4d4-4874-8d73-de7dc1fc1928 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.528237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4113747-b4d4-4874-8d73-de7dc1fc1928 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.528237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4113747-b4d4-4874-8d73-de7dc1fc1928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.532905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 788/1564] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:23.533745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d841a-7552-4c05-a5f2-52c94efb25ca None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.543893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d841a-7552-4c05-a5f2-52c94efb25ca None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.548374 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b4113747-b4d4-4874-8d73-de7dc1fc1928 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:23.549446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8f915-228b-4f4a-bd1e-3c3d792db7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.549601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b4113747-b4d4-4874-8d73-de7dc1fc1928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:23.550280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8f915-228b-4f4a-bd1e-3c3d792db7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.550633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8f915-228b-4f4a-bd1e-3c3d792db7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.550975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8f915-228b-4f4a-bd1e-3c3d792db7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.551770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8f915-228b-4f4a-bd1e-3c3d792db7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.553422 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8f915-228b-4f4a-bd1e-3c3d792db7e1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.557669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 777/1565] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:23.558425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4113747-b4d4-4874-8d73-de7dc1fc1928 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.564649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 789/1566] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:23.567003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5ac00-ff86-456c-9699-fdf42a0765e2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.567290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5ac00-ff86-456c-9699-fdf42a0765e2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.567626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5ac00-ff86-456c-9699-fdf42a0765e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.568985 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4113747-b4d4-4874-8d73-de7dc1fc1928 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.573576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530bc5c-44ec-4a01-9497-91c176ef9452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.574270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530bc5c-44ec-4a01-9497-91c176ef9452 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.574557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530bc5c-44ec-4a01-9497-91c176ef9452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.574833 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530bc5c-44ec-4a01-9497-91c176ef9452 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.576303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530bc5c-44ec-4a01-9497-91c176ef9452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.577450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530bc5c-44ec-4a01-9497-91c176ef9452 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.586536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5ab5ac00-ff86-456c-9699-fdf42a0765e2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:23.588109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5ab5ac00-ff86-456c-9699-fdf42a0765e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:23.596293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 790/1567] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:23.597072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5ac00-ff86-456c-9699-fdf42a0765e2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.602807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 778/1568] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/users => generated 283 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:23.604048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd1d8b-225f-434f-8055-a179d00d7db1 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.604411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd1d8b-225f-434f-8055-a179d00d7db1 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.604760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd1d8b-225f-434f-8055-a179d00d7db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.606843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5ac00-ff86-456c-9699-fdf42a0765e2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.612216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed68bb-794e-4b1f-8cc0-172ed53cae5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.612853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed68bb-794e-4b1f-8cc0-172ed53cae5c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.613130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed68bb-794e-4b1f-8cc0-172ed53cae5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.613403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed68bb-794e-4b1f-8cc0-172ed53cae5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.614921 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed68bb-794e-4b1f-8cc0-172ed53cae5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:23.614921 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed68bb-794e-4b1f-8cc0-172ed53cae5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.616014 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed68bb-794e-4b1f-8cc0-172ed53cae5c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.625278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-52cd1d8b-225f-434f-8055-a179d00d7db1 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:23.626512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-52cd1d8b-225f-434f-8055-a179d00d7db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:23.628366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 791/1569] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:39:23.629158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed68bb-794e-4b1f-8cc0-172ed53cae5c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.639654 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 779/1570] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:23.640392 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd1d8b-225f-434f-8055-a179d00d7db1 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.640655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed68bb-794e-4b1f-8cc0-172ed53cae5c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.645772 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0844880-b7da-4772-95e2-b78e828069ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.646451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0844880-b7da-4772-95e2-b78e828069ea admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.646738 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0844880-b7da-4772-95e2-b78e828069ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.647016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0844880-b7da-4772-95e2-b78e828069ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.647740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0844880-b7da-4772-95e2-b78e828069ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:23.648050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0844880-b7da-4772-95e2-b78e828069ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.649642 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0844880-b7da-4772-95e2-b78e828069ea admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.661652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 792/1571] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:39:23.662500 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0844880-b7da-4772-95e2-b78e828069ea admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.672330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0844880-b7da-4772-95e2-b78e828069ea admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.676907 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793d362-240d-4d9d-81dd-60f6d6aaf27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.677534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793d362-240d-4d9d-81dd-60f6d6aaf27c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.677826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793d362-240d-4d9d-81dd-60f6d6aaf27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.678139 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793d362-240d-4d9d-81dd-60f6d6aaf27c admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.678859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793d362-240d-4d9d-81dd-60f6d6aaf27c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:23.679183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793d362-240d-4d9d-81dd-60f6d6aaf27c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.680741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793d362-240d-4d9d-81dd-60f6d6aaf27c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.692632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 793/1572] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:39:23.698960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793d362-240d-4d9d-81dd-60f6d6aaf27c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.699936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd1d8b-225f-434f-8055-a179d00d7db1 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.703726 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f7d42-86d4-4939-9cc1-c752cc1a5d38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.704376 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f7d42-86d4-4939-9cc1-c752cc1a5d38 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.704659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f7d42-86d4-4939-9cc1-c752cc1a5d38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.704931 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f7d42-86d4-4939-9cc1-c752cc1a5d38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.708791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793d362-240d-4d9d-81dd-60f6d6aaf27c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.710843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:23.713905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a84858-0c9d-4e4d-9d2f-e8d3250b72bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.714577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a84858-0c9d-4e4d-9d2f-e8d3250b72bf admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.714682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f7d42-86d4-4939-9cc1-c752cc1a5d38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.714863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a84858-0c9d-4e4d-9d2f-e8d3250b72bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.715134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a84858-0c9d-4e4d-9d2f-e8d3250b72bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.715847 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a84858-0c9d-4e4d-9d2f-e8d3250b72bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.716568 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f7d42-86d4-4939-9cc1-c752cc1a5d38 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.717437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a84858-0c9d-4e4d-9d2f-e8d3250b72bf admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.727183 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 780/1573] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:23.731254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 794/1574] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:23.731392 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f7d42-86d4-4939-9cc1-c752cc1a5d38 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.731520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a84858-0c9d-4e4d-9d2f-e8d3250b72bf admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.740983 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a84858-0c9d-4e4d-9d2f-e8d3250b72bf admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.742041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f7d42-86d4-4939-9cc1-c752cc1a5d38 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.745414 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936933b2-e910-4bc2-a6e5-f79e361e2413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.746099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936933b2-e910-4bc2-a6e5-f79e361e2413 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.746406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936933b2-e910-4bc2-a6e5-f79e361e2413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.746678 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936933b2-e910-4bc2-a6e5-f79e361e2413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.747391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936933b2-e910-4bc2-a6e5-f79e361e2413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.747783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb89c70-1dd1-4c55-8ca3-3d38c4a88176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.748779 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb89c70-1dd1-4c55-8ca3-3d38c4a88176 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.748897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb89c70-1dd1-4c55-8ca3-3d38c4a88176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.748987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936933b2-e910-4bc2-a6e5-f79e361e2413 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.750163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb89c70-1dd1-4c55-8ca3-3d38c4a88176 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.751028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb89c70-1dd1-4c55-8ca3-3d38c4a88176 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:23.751448 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb89c70-1dd1-4c55-8ca3-3d38c4a88176 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.753541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb89c70-1dd1-4c55-8ca3-3d38c4a88176 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.763717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 795/1575] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:23.763717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936933b2-e910-4bc2-a6e5-f79e361e2413 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.773473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 781/1576] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:39:23.773473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb89c70-1dd1-4c55-8ca3-3d38c4a88176 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.779228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb89c70-1dd1-4c55-8ca3-3d38c4a88176 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.779398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936933b2-e910-4bc2-a6e5-f79e361e2413 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.784766 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507d174-8d51-4bba-a740-766f62411e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.785502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507d174-8d51-4bba-a740-766f62411e73 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.785853 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507d174-8d51-4bba-a740-766f62411e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.786224 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507d174-8d51-4bba-a740-766f62411e73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.787022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507d174-8d51-4bba-a740-766f62411e73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:23.787400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507d174-8d51-4bba-a740-766f62411e73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.787534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f9795-2115-47cf-b081-1e4fd3864a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.788078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f9795-2115-47cf-b081-1e4fd3864a7b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.788410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f9795-2115-47cf-b081-1e4fd3864a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.788689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f9795-2115-47cf-b081-1e4fd3864a7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.789297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507d174-8d51-4bba-a740-766f62411e73 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.789417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f9795-2115-47cf-b081-1e4fd3864a7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.791344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f9795-2115-47cf-b081-1e4fd3864a7b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.797981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:23.803954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 782/1577] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:39:23.815999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507d174-8d51-4bba-a740-766f62411e73 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.827664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507d174-8d51-4bba-a740-766f62411e73 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.828848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 796/1578] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:23.833070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f9795-2115-47cf-b081-1e4fd3864a7b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.897893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f805c-8c52-4892-a845-3d2eae3da09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.899036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f805c-8c52-4892-a845-3d2eae3da09a admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.899325 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f805c-8c52-4892-a845-3d2eae3da09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.899602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f805c-8c52-4892-a845-3d2eae3da09a admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.900548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f805c-8c52-4892-a845-3d2eae3da09a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.903077 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f805c-8c52-4892-a845-3d2eae3da09a admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.906291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f9795-2115-47cf-b081-1e4fd3864a7b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:23.911755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e90b00-cec7-4e86-9ae5-d28052911de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:23.912426 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e90b00-cec7-4e86-9ae5-d28052911de6 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:23.912713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e90b00-cec7-4e86-9ae5-d28052911de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:23.912990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e90b00-cec7-4e86-9ae5-d28052911de6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:23.914170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e90b00-cec7-4e86-9ae5-d28052911de6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:23.916724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e90b00-cec7-4e86-9ae5-d28052911de6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:23.938808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 783/1579] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:23.938808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f805c-8c52-4892-a845-3d2eae3da09a admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:23.964509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 797/1580] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:23 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:23.965867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e90b00-cec7-4e86-9ae5-d28052911de6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.007728 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f805c-8c52-4892-a845-3d2eae3da09a admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.049786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e90b00-cec7-4e86-9ae5-d28052911de6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.065152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46479796-9ab6-4365-8666-8d0d5990369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.065962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46479796-9ab6-4365-8666-8d0d5990369f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.066338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46479796-9ab6-4365-8666-8d0d5990369f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.066729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46479796-9ab6-4365-8666-8d0d5990369f admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.067618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46479796-9ab6-4365-8666-8d0d5990369f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.069304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46479796-9ab6-4365-8666-8d0d5990369f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.076798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326eb405-37ca-4232-8acf-3efc85a96bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.077797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326eb405-37ca-4232-8acf-3efc85a96bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.078186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326eb405-37ca-4232-8acf-3efc85a96bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.078522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326eb405-37ca-4232-8acf-3efc85a96bb3 admin admin] PATH_INFO: `/v3/projects/df3ed3c23dde49b382ce9016e421e4de/users/1bc5799ac34349518854829ec2c259f9/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.081090 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:24.091896 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 798/1581] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:24.094894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46479796-9ab6-4365-8666-8d0d5990369f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.097455 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326eb405-37ca-4232-8acf-3efc85a96bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3ed3c23dde49b382ce9016e421e4de, user_id=1bc5799ac34349518854829ec2c259f9, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.100286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326eb405-37ca-4232-8acf-3efc85a96bb3 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.111866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46479796-9ab6-4365-8666-8d0d5990369f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.118030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 784/1582] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:23 2022] PUT /identity/v3/projects/df3ed3c23dde49b382ce9016e421e4de/users/1bc5799ac34349518854829ec2c259f9/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:24.119012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326eb405-37ca-4232-8acf-3efc85a96bb3 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.195593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326eb405-37ca-4232-8acf-3efc85a96bb3 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.201239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327df45-4a35-49d0-8c17-a1faa5101df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.202734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327df45-4a35-49d0-8c17-a1faa5101df4 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.203027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327df45-4a35-49d0-8c17-a1faa5101df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.203306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327df45-4a35-49d0-8c17-a1faa5101df4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.204010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327df45-4a35-49d0-8c17-a1faa5101df4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.206074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327df45-4a35-49d0-8c17-a1faa5101df4 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.206220 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7be02-802d-40be-a319-a05ac86e5e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.207362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7be02-802d-40be-a319-a05ac86e5e27 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.207667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7be02-802d-40be-a319-a05ac86e5e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.207951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7be02-802d-40be-a319-a05ac86e5e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.208796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7be02-802d-40be-a319-a05ac86e5e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.210747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7be02-802d-40be-a319-a05ac86e5e27 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.229423 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:24.245420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 785/1583] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:24 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:24.245420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327df45-4a35-49d0-8c17-a1faa5101df4 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.251439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 799/1584] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:24 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:24.251439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7be02-802d-40be-a319-a05ac86e5e27 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.309785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327df45-4a35-49d0-8c17-a1faa5101df4 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.324482 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7be02-802d-40be-a319-a05ac86e5e27 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.330689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb2ce0-d292-4908-8a9d-5b74d568cf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.331372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb2ce0-d292-4908-8a9d-5b74d568cf43 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.331831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb2ce0-d292-4908-8a9d-5b74d568cf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.335537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb2ce0-d292-4908-8a9d-5b74d568cf43 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.335537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb2ce0-d292-4908-8a9d-5b74d568cf43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.335537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb2ce0-d292-4908-8a9d-5b74d568cf43 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.363011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 800/1585] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:24 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:24.363517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb2ce0-d292-4908-8a9d-5b74d568cf43 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.376041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb2ce0-d292-4908-8a9d-5b74d568cf43 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.381239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847d023-df6c-4f0b-8dd7-b051a510522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.382055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847d023-df6c-4f0b-8dd7-b051a510522d admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.382409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847d023-df6c-4f0b-8dd7-b051a510522d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.382751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847d023-df6c-4f0b-8dd7-b051a510522d admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.383451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847d023-df6c-4f0b-8dd7-b051a510522d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.383671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08c9c2-da82-492b-8398-a206d4baf2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.384275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08c9c2-da82-492b-8398-a206d4baf2e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.384633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08c9c2-da82-492b-8398-a206d4baf2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.384843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08c9c2-da82-492b-8398-a206d4baf2e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.385532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08c9c2-da82-492b-8398-a206d4baf2e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.385831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847d023-df6c-4f0b-8dd7-b051a510522d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.387512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08c9c2-da82-492b-8398-a206d4baf2e4 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.418663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 786/1586] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:24 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:24.419283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 801/1587] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:24 2022] POST /identity/v3/users => generated 287 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:24.419942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08c9c2-da82-492b-8398-a206d4baf2e4 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.420955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a51755-a141-4ae1-a31b-292a99089585 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.421478 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a51755-a141-4ae1-a31b-292a99089585 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.421807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a51755-a141-4ae1-a31b-292a99089585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.428480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:24.434608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08c9c2-da82-492b-8398-a206d4baf2e4 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.441957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0594f92-b9bf-4a4b-8600-3d14bd571c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.443424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0594f92-b9bf-4a4b-8600-3d14bd571c93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.444109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0594f92-b9bf-4a4b-8600-3d14bd571c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.444658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0594f92-b9bf-4a4b-8600-3d14bd571c93 admin admin] PATH_INFO: `/v3/projects/2bcc5a12b5ef476d944b824008f03014/users/cdfc9e7c0c7441e5b643ffa8083d8c37/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.447181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f0a51755-a141-4ae1-a31b-292a99089585 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:24.448435 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f0a51755-a141-4ae1-a31b-292a99089585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:24.464805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 802/1588] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:24.465591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a51755-a141-4ae1-a31b-292a99089585 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.466873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0594f92-b9bf-4a4b-8600-3d14bd571c93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bcc5a12b5ef476d944b824008f03014, user_id=cdfc9e7c0c7441e5b643ffa8083d8c37, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.473411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0594f92-b9bf-4a4b-8600-3d14bd571c93 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.480442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a51755-a141-4ae1-a31b-292a99089585 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.489251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea97709-67ee-4dc5-94d0-22c8106c1826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.489758 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97709-67ee-4dc5-94d0-22c8106c1826 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.491438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97709-67ee-4dc5-94d0-22c8106c1826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.491579 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97709-67ee-4dc5-94d0-22c8106c1826 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.492207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea97709-67ee-4dc5-94d0-22c8106c1826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.504700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 787/1589] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:24 2022] PUT /identity/v3/projects/2bcc5a12b5ef476d944b824008f03014/users/cdfc9e7c0c7441e5b643ffa8083d8c37/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:24.504700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0594f92-b9bf-4a4b-8600-3d14bd571c93 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.505023 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea97709-67ee-4dc5-94d0-22c8106c1826 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.532877 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 803/1590] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:24 2022] POST /identity/v3/users => generated 293 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:24.534162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea97709-67ee-4dc5-94d0-22c8106c1826 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.546999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea97709-67ee-4dc5-94d0-22c8106c1826 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.591954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0594f92-b9bf-4a4b-8600-3d14bd571c93 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.598245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cda3e9-3f94-4c80-8957-04a3db26f437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.599064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cda3e9-3f94-4c80-8957-04a3db26f437 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.599360 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cda3e9-3f94-4c80-8957-04a3db26f437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.599642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cda3e9-3f94-4c80-8957-04a3db26f437 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.600353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cda3e9-3f94-4c80-8957-04a3db26f437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.602333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cda3e9-3f94-4c80-8957-04a3db26f437 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.626613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 788/1591] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:24 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:24.630721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cda3e9-3f94-4c80-8957-04a3db26f437 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.635675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b3d68-f367-4d73-a9bf-287767afc540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.636759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b3d68-f367-4d73-a9bf-287767afc540 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.637217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b3d68-f367-4d73-a9bf-287767afc540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.637747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b3d68-f367-4d73-a9bf-287767afc540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.638567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b3d68-f367-4d73-a9bf-287767afc540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.641142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b3d68-f367-4d73-a9bf-287767afc540 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.642803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cda3e9-3f94-4c80-8957-04a3db26f437 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.649861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e7b87-9dfc-4b80-984a-b2eb29193f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.651068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e7b87-9dfc-4b80-984a-b2eb29193f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.651571 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e7b87-9dfc-4b80-984a-b2eb29193f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.652035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e7b87-9dfc-4b80-984a-b2eb29193f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.652962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e7b87-9dfc-4b80-984a-b2eb29193f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.656263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 804/1592] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:24.656263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b3d68-f367-4d73-a9bf-287767afc540 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.658485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e7b87-9dfc-4b80-984a-b2eb29193f27 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.670821 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 789/1593] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:24.672215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e7b87-9dfc-4b80-984a-b2eb29193f27 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.673720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b3d68-f367-4d73-a9bf-287767afc540 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.674414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-dd5e7b87-9dfc-4b80-984a-b2eb29193f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:24.679266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f2488-2f4d-4898-b0a6-18c23b836fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.680573 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f2488-2f4d-4898-b0a6-18c23b836fee admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.680916 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f2488-2f4d-4898-b0a6-18c23b836fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.681252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f2488-2f4d-4898-b0a6-18c23b836fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.682137 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f2488-2f4d-4898-b0a6-18c23b836fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.685111 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f2488-2f4d-4898-b0a6-18c23b836fee admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.706240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 805/1594] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:24.707665 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04604275-0514-4d1a-8e1e-f20506c0b67b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.707960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04604275-0514-4d1a-8e1e-f20506c0b67b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.708380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04604275-0514-4d1a-8e1e-f20506c0b67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.744129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-04604275-0514-4d1a-8e1e-f20506c0b67b None None] MFA Rules not processed for user `cdfc9e7c0c7441e5b643ffa8083d8c37`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:24.745525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-04604275-0514-4d1a-8e1e-f20506c0b67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:24.748734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e7b87-9dfc-4b80-984a-b2eb29193f27 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.755111 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3792b-6619-4c79-8007-446de976f5c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.756273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3792b-6619-4c79-8007-446de976f5c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.756714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3792b-6619-4c79-8007-446de976f5c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.757286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3792b-6619-4c79-8007-446de976f5c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.761829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:INSERT=10,keystone:SELECT=90 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:24.773970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3792b-6619-4c79-8007-446de976f5c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.775594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3792b-6619-4c79-8007-446de976f5c3 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.787482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 790/1595] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:24.788575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3792b-6619-4c79-8007-446de976f5c3 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.799552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3792b-6619-4c79-8007-446de976f5c3 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.814294 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bc524-876a-4605-95ec-59d53a0c0ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.815022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bc524-876a-4605-95ec-59d53a0c0ada admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.815310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bc524-876a-4605-95ec-59d53a0c0ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.815591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bc524-876a-4605-95ec-59d53a0c0ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.816614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bc524-876a-4605-95ec-59d53a0c0ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.818606 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bc524-876a-4605-95ec-59d53a0c0ada admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.828834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 806/1596] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:24.829819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04604275-0514-4d1a-8e1e-f20506c0b67b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.843649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04604275-0514-4d1a-8e1e-f20506c0b67b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.848535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 791/1597] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:24.850673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bc524-876a-4605-95ec-59d53a0c0ada admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.853187 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f0e62-ec9e-4849-bec6-ec8db34d4c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.854085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f0e62-ec9e-4849-bec6-ec8db34d4c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.854473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f0e62-ec9e-4849-bec6-ec8db34d4c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.854936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f0e62-ec9e-4849-bec6-ec8db34d4c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.856025 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f0e62-ec9e-4849-bec6-ec8db34d4c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.857755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f0e62-ec9e-4849-bec6-ec8db34d4c4f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.872273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 807/1598] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:24.873574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f0e62-ec9e-4849-bec6-ec8db34d4c4f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.874929 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bc524-876a-4605-95ec-59d53a0c0ada admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.888413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f0e62-ec9e-4849-bec6-ec8db34d4c4f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.892934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad95ff9-0b89-46d2-acc9-eb9f13298328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.893638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad95ff9-0b89-46d2-acc9-eb9f13298328 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.894191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad95ff9-0b89-46d2-acc9-eb9f13298328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.894315 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad95ff9-0b89-46d2-acc9-eb9f13298328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.895058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad95ff9-0b89-46d2-acc9-eb9f13298328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.897434 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6437d06-e18c-4ced-901d-a43a37a8496a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.898492 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6437d06-e18c-4ced-901d-a43a37a8496a admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.898992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6437d06-e18c-4ced-901d-a43a37a8496a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.899478 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6437d06-e18c-4ced-901d-a43a37a8496a admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.900497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6437d06-e18c-4ced-901d-a43a37a8496a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.902039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad95ff9-0b89-46d2-acc9-eb9f13298328 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.903997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6437d06-e18c-4ced-901d-a43a37a8496a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.924783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 808/1599] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:24.924783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6437d06-e18c-4ced-901d-a43a37a8496a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.926650 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 792/1600] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:24.928001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad95ff9-0b89-46d2-acc9-eb9f13298328 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:24.938430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6437d06-e18c-4ced-901d-a43a37a8496a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:24.945063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e5e7-ed75-4814-9a9e-69e3d823ef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:24.946335 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e5e7-ed75-4814-9a9e-69e3d823ef37 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:24.947084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e5e7-ed75-4814-9a9e-69e3d823ef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:24.947450 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e5e7-ed75-4814-9a9e-69e3d823ef37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:24.948437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154e5e7-ed75-4814-9a9e-69e3d823ef37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:24.950580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154e5e7-ed75-4814-9a9e-69e3d823ef37 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:24.968432 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 809/1601] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:24.969588 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e5e7-ed75-4814-9a9e-69e3d823ef37 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:25.003722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e5e7-ed75-4814-9a9e-69e3d823ef37 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:25.010333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9c21a-cc73-4e59-9ed0-3cdd738e6f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:25.011386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9c21a-cc73-4e59-9ed0-3cdd738e6f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.011524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9c21a-cc73-4e59-9ed0-3cdd738e6f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.012065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9c21a-cc73-4e59-9ed0-3cdd738e6f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.012868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9c21a-cc73-4e59-9ed0-3cdd738e6f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:25.014846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9c21a-cc73-4e59-9ed0-3cdd738e6f40 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:25.015991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:25.031935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 810/1602] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:25.037001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9c21a-cc73-4e59-9ed0-3cdd738e6f40 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:25.042793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad95ff9-0b89-46d2-acc9-eb9f13298328 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:25.052696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9c21a-cc73-4e59-9ed0-3cdd738e6f40 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:25.054140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a5c9c-5bfe-4960-bed9-10dad55ffb6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:25.055025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a5c9c-5bfe-4960-bed9-10dad55ffb6e service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.055413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a5c9c-5bfe-4960-bed9-10dad55ffb6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.056481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a5c9c-5bfe-4960-bed9-10dad55ffb6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.067955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9a46d-8da2-4096-9829-615e572c5fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:25.068837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9a46d-8da2-4096-9829-615e572c5fcc admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.070525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9a46d-8da2-4096-9829-615e572c5fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.071464 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9a46d-8da2-4096-9829-615e572c5fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.073940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9a46d-8da2-4096-9829-615e572c5fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:25.075955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9a46d-8da2-4096-9829-615e572c5fcc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:25.082056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a5c9c-5bfe-4960-bed9-10dad55ffb6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:25.087512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a5c9c-5bfe-4960-bed9-10dad55ffb6e service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:25.111087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 811/1603] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:25.111087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9a46d-8da2-4096-9829-615e572c5fcc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:25.116371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 793/1604] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:25.119524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a5c9c-5bfe-4960-bed9-10dad55ffb6e service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:25.136275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9a46d-8da2-4096-9829-615e572c5fcc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:25.149172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a5c9c-5bfe-4960-bed9-10dad55ffb6e service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:25.150131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad2d15-1850-4197-ada3-806982111447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:25.152301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad2d15-1850-4197-ada3-806982111447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.153653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad2d15-1850-4197-ada3-806982111447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.154422 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad2d15-1850-4197-ada3-806982111447 admin admin] PATH_INFO: `/v3/projects/cb082229375f42198cba0b84485bd7f5/users/f089b2feef6a4b8599dd077ea809a25a/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.162473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d87cc-938d-48ed-a9e3-b373bfd5df75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:25.163546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d87cc-938d-48ed-a9e3-b373bfd5df75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.165444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d87cc-938d-48ed-a9e3-b373bfd5df75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.166091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d87cc-938d-48ed-a9e3-b373bfd5df75 admin admin] PATH_INFO: `/v3/projects/43179fa28eb0451ca665d1d6b0e70b73/users/996fbc75a3554253a9dec73ac8dc6f93/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.193691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad2d15-1850-4197-ada3-806982111447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb082229375f42198cba0b84485bd7f5, user_id=f089b2feef6a4b8599dd077ea809a25a, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:25.195155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-13ad2d15-1850-4197-ada3-806982111447 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:39:25.195155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 25 12:39:25.195155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: Jul 25 12:39:25.201421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad2d15-1850-4197-ada3-806982111447 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:25.206662 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d87cc-938d-48ed-a9e3-b373bfd5df75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43179fa28eb0451ca665d1d6b0e70b73, user_id=996fbc75a3554253a9dec73ac8dc6f93, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:25.210554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d87cc-938d-48ed-a9e3-b373bfd5df75 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:25.247477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 812/1605] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:25 2022] PUT /identity/v3/projects/cb082229375f42198cba0b84485bd7f5/users/f089b2feef6a4b8599dd077ea809a25a/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:25.247477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad2d15-1850-4197-ada3-806982111447 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:25.256874 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 794/1606] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:25 2022] PUT /identity/v3/projects/43179fa28eb0451ca665d1d6b0e70b73/users/996fbc75a3554253a9dec73ac8dc6f93/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:25.258983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d87cc-938d-48ed-a9e3-b373bfd5df75 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:25.405083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad2d15-1850-4197-ada3-806982111447 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:25.406293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d87cc-938d-48ed-a9e3-b373bfd5df75 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:25.411439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba685a-1f4a-42e5-b9ad-f97628d39a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:25.412844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba685a-1f4a-42e5-b9ad-f97628d39a60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.413338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba685a-1f4a-42e5-b9ad-f97628d39a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.413832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba685a-1f4a-42e5-b9ad-f97628d39a60 admin admin] PATH_INFO: `/v3/projects/7dfb4873fce0400b88e139274d392e58/users/cc58762391d1495187ad44a7f9c0d64f/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.415995 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1f106-f48b-4406-abd5-b0bfebdae88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:25.417088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1f106-f48b-4406-abd5-b0bfebdae88c admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.417547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1f106-f48b-4406-abd5-b0bfebdae88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.418002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1f106-f48b-4406-abd5-b0bfebdae88c admin admin] PATH_INFO: `/v3/projects/49256bcb551a4f8680db9a73763c435e/users/6f611c60491941ca8a0200212ab0c7a7/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.449138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1f106-f48b-4406-abd5-b0bfebdae88c admin admin] RBAC: Authorizing `identity:create_grant(project_id=49256bcb551a4f8680db9a73763c435e, user_id=6f611c60491941ca8a0200212ab0c7a7, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:25.453481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1f106-f48b-4406-abd5-b0bfebdae88c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:25.454536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba685a-1f4a-42e5-b9ad-f97628d39a60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dfb4873fce0400b88e139274d392e58, user_id=cc58762391d1495187ad44a7f9c0d64f, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:25.459662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba685a-1f4a-42e5-b9ad-f97628d39a60 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:25.491834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 813/1607] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:25 2022] PUT /identity/v3/projects/7dfb4873fce0400b88e139274d392e58/users/cc58762391d1495187ad44a7f9c0d64f/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:25.491834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba685a-1f4a-42e5-b9ad-f97628d39a60 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:25.502276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 795/1608] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:25 2022] PUT /identity/v3/projects/49256bcb551a4f8680db9a73763c435e/users/6f611c60491941ca8a0200212ab0c7a7/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:25.502276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a336ac-60a2-4a19-b15d-89f66c100998 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.502276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a336ac-60a2-4a19-b15d-89f66c100998 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.502276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a336ac-60a2-4a19-b15d-89f66c100998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.542286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:25.574713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-41a336ac-60a2-4a19-b15d-89f66c100998 None None] MFA Rules not processed for user `f089b2feef6a4b8599dd077ea809a25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:25.576821 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-41a336ac-60a2-4a19-b15d-89f66c100998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:25.597818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba685a-1f4a-42e5-b9ad-f97628d39a60 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:25.604482 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808965fd-2fb8-4bf2-8e0f-40b56a3bf1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:25.605395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808965fd-2fb8-4bf2-8e0f-40b56a3bf1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.605680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808965fd-2fb8-4bf2-8e0f-40b56a3bf1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.606019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808965fd-2fb8-4bf2-8e0f-40b56a3bf1b3 admin admin] PATH_INFO: `/v3/projects/c24e32d2fb6e4a3db413acc06528d15e/users/b0f3576844744c538a4755221c4added/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.627384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808965fd-2fb8-4bf2-8e0f-40b56a3bf1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c24e32d2fb6e4a3db413acc06528d15e, user_id=b0f3576844744c538a4755221c4added, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:25.630302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808965fd-2fb8-4bf2-8e0f-40b56a3bf1b3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:25.653214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 814/1609] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:25 2022] PUT /identity/v3/projects/c24e32d2fb6e4a3db413acc06528d15e/users/b0f3576844744c538a4755221c4added/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:25.655021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4d4db-d988-4dc2-90dd-d1c259a2c54d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.655441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4d4db-d988-4dc2-90dd-d1c259a2c54d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.655736 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4d4db-d988-4dc2-90dd-d1c259a2c54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.672543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 796/1610] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:25.674687 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22daa5-34bd-40fc-b30f-40c5437b983b None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.675130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22daa5-34bd-40fc-b30f-40c5437b983b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.675578 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22daa5-34bd-40fc-b30f-40c5437b983b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.694939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-82f4d4db-d988-4dc2-90dd-d1c259a2c54d None None] MFA Rules not processed for user `996fbc75a3554253a9dec73ac8dc6f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:25.696796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-82f4d4db-d988-4dc2-90dd-d1c259a2c54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:25.701955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-7c22daa5-34bd-40fc-b30f-40c5437b983b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:25.704701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7c22daa5-34bd-40fc-b30f-40c5437b983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:25.752855 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:25.814955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 797/1611] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:25.816437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a882e-308a-4ef0-8319-d5673ebca20b None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.816730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a882e-308a-4ef0-8319-d5673ebca20b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.817006 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a882e-308a-4ef0-8319-d5673ebca20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.877620 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 815/1612] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:25.877620 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5952535-9d58-49c4-8b0c-ce56864486d8 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.877620 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5952535-9d58-49c4-8b0c-ce56864486d8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.877620 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5952535-9d58-49c4-8b0c-ce56864486d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.877620 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a5952535-9d58-49c4-8b0c-ce56864486d8 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:25.877620 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a5952535-9d58-49c4-8b0c-ce56864486d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:25.879579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-875a882e-308a-4ef0-8319-d5673ebca20b None None] MFA Rules not processed for user `cc58762391d1495187ad44a7f9c0d64f`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:25.879579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-875a882e-308a-4ef0-8319-d5673ebca20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:25.892335 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 816/1613] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:25.896500 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5952535-9d58-49c4-8b0c-ce56864486d8 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:25.918368 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5952535-9d58-49c4-8b0c-ce56864486d8 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:25.925897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7756c46-e0b7-4015-9a2f-89dabec6e364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:25.927382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7756c46-e0b7-4015-9a2f-89dabec6e364 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:25.927382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7756c46-e0b7-4015-9a2f-89dabec6e364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:25.927568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7756c46-e0b7-4015-9a2f-89dabec6e364 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:25.928375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7756c46-e0b7-4015-9a2f-89dabec6e364 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:25.931240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7756c46-e0b7-4015-9a2f-89dabec6e364 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:26.013425 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 817/1614] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:25 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:26.015563 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7756c46-e0b7-4015-9a2f-89dabec6e364 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:26.044551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 798/1615] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:26.047201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a882e-308a-4ef0-8319-d5673ebca20b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:26.159189 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7756c46-e0b7-4015-9a2f-89dabec6e364 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:26.165316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ba5f2-d685-4760-ba93-8a098bbdfbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:26.170004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ba5f2-d685-4760-ba93-8a098bbdfbcf admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:26.170004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ba5f2-d685-4760-ba93-8a098bbdfbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:26.170004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ba5f2-d685-4760-ba93-8a098bbdfbcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:26.170778 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ba5f2-d685-4760-ba93-8a098bbdfbcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:26.174131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ba5f2-d685-4760-ba93-8a098bbdfbcf admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:26.185523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a882e-308a-4ef0-8319-d5673ebca20b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:26.186347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:26.215215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 818/1616] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:26 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:26.216100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ba5f2-d685-4760-ba93-8a098bbdfbcf admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:26.305661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d232fe9-ee80-412d-9aa3-42cfd68ec6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:26.306781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d232fe9-ee80-412d-9aa3-42cfd68ec6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:26.307305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d232fe9-ee80-412d-9aa3-42cfd68ec6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:26.307789 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d232fe9-ee80-412d-9aa3-42cfd68ec6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:26.332241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d232fe9-ee80-412d-9aa3-42cfd68ec6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:26.333854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d232fe9-ee80-412d-9aa3-42cfd68ec6b3 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:26.355591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ba5f2-d685-4760-ba93-8a098bbdfbcf admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:26.360491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 799/1617] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:26.361343 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d232fe9-ee80-412d-9aa3-42cfd68ec6b3 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:26.369294 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56d4ef-20bb-4da4-a62e-d53c0ff8c73f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:26.370014 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56d4ef-20bb-4da4-a62e-d53c0ff8c73f service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:26.370365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56d4ef-20bb-4da4-a62e-d53c0ff8c73f service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:26.370660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56d4ef-20bb-4da4-a62e-d53c0ff8c73f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:26.380459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d232fe9-ee80-412d-9aa3-42cfd68ec6b3 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:26.418527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:26.501220 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fa38f-2e91-4365-b999-9ca8a56f8151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:26.501953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fa38f-2e91-4365-b999-9ca8a56f8151 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:26.502237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fa38f-2e91-4365-b999-9ca8a56f8151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:26.502516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fa38f-2e91-4365-b999-9ca8a56f8151 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:26.503262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fa38f-2e91-4365-b999-9ca8a56f8151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:26.507960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fa38f-2e91-4365-b999-9ca8a56f8151 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:26.529086 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56d4ef-20bb-4da4-a62e-d53c0ff8c73f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:26.531135 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56d4ef-20bb-4da4-a62e-d53c0ff8c73f service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:26.549961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 819/1618] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:26.553218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56d4ef-20bb-4da4-a62e-d53c0ff8c73f service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:26.559558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 800/1619] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:26 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:26.572952 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fa38f-2e91-4365-b999-9ca8a56f8151 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:26.574972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56d4ef-20bb-4da4-a62e-d53c0ff8c73f service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:26.583285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef312f3c-8c1f-45c2-8ada-994c46e494d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:26.584736 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef312f3c-8c1f-45c2-8ada-994c46e494d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:26.585111 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef312f3c-8c1f-45c2-8ada-994c46e494d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:26.585565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef312f3c-8c1f-45c2-8ada-994c46e494d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:26.586093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fa38f-2e91-4365-b999-9ca8a56f8151 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:26.586930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef312f3c-8c1f-45c2-8ada-994c46e494d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:26.589991 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef312f3c-8c1f-45c2-8ada-994c46e494d7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:26.598632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d20047-f297-4b35-b0fa-2e2a2ab63ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:26.599336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d20047-f297-4b35-b0fa-2e2a2ab63ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:26.599621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d20047-f297-4b35-b0fa-2e2a2ab63ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:26.599900 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d20047-f297-4b35-b0fa-2e2a2ab63ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:26.600605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d20047-f297-4b35-b0fa-2e2a2ab63ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:26.602591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d20047-f297-4b35-b0fa-2e2a2ab63ef4 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:26.642079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 820/1620] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:26 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:26.643175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef312f3c-8c1f-45c2-8ada-994c46e494d7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:26.668378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 801/1621] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:26 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:26.668378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d20047-f297-4b35-b0fa-2e2a2ab63ef4 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:26.685409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d20047-f297-4b35-b0fa-2e2a2ab63ef4 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:26.786465 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c025bf8-746c-4f37-bcab-98cb75155e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:26.788148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c025bf8-746c-4f37-bcab-98cb75155e3c admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:26.788523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c025bf8-746c-4f37-bcab-98cb75155e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:26.788933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c025bf8-746c-4f37-bcab-98cb75155e3c admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:26.789983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c025bf8-746c-4f37-bcab-98cb75155e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:26.793165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c025bf8-746c-4f37-bcab-98cb75155e3c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:26.815530 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef312f3c-8c1f-45c2-8ada-994c46e494d7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:26.829238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce0edc-5b5e-40f7-ace4-5adf58e354e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:26.829967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce0edc-5b5e-40f7-ace4-5adf58e354e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:26.830869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce0edc-5b5e-40f7-ace4-5adf58e354e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:26.830869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce0edc-5b5e-40f7-ace4-5adf58e354e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:26.831251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce0edc-5b5e-40f7-ace4-5adf58e354e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:26.836354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 802/1622] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:26 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:26.836532 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce0edc-5b5e-40f7-ace4-5adf58e354e6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:26.837512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c025bf8-746c-4f37-bcab-98cb75155e3c admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:26.849103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c025bf8-746c-4f37-bcab-98cb75155e3c admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:26.856926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 821/1623] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:26.860947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce0edc-5b5e-40f7-ace4-5adf58e354e6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:26.862246 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79140f-6ed6-47e7-9eec-65e4eaca9551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:26.863045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79140f-6ed6-47e7-9eec-65e4eaca9551 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:26.863331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79140f-6ed6-47e7-9eec-65e4eaca9551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:26.863613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79140f-6ed6-47e7-9eec-65e4eaca9551 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:26.864560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79140f-6ed6-47e7-9eec-65e4eaca9551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:26.866244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79140f-6ed6-47e7-9eec-65e4eaca9551 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:26.883434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 803/1624] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:26.884330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79140f-6ed6-47e7-9eec-65e4eaca9551 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:26.886622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce0edc-5b5e-40f7-ace4-5adf58e354e6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:26.895245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79140f-6ed6-47e7-9eec-65e4eaca9551 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:26.898812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3289fe0-3760-4cdf-bbc7-ca44072b4468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:26.899501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3289fe0-3760-4cdf-bbc7-ca44072b4468 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:26.899787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3289fe0-3760-4cdf-bbc7-ca44072b4468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:26.900064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3289fe0-3760-4cdf-bbc7-ca44072b4468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:26.900801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3289fe0-3760-4cdf-bbc7-ca44072b4468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:26.901705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad5d76-1bcf-4789-ad25-a2d721cb5af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:26.902457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3289fe0-3760-4cdf-bbc7-ca44072b4468 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:26.902673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad5d76-1bcf-4789-ad25-a2d721cb5af7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:26.902928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad5d76-1bcf-4789-ad25-a2d721cb5af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:26.903206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad5d76-1bcf-4789-ad25-a2d721cb5af7 admin admin] PATH_INFO: `/v3/projects/47512038402f4e7fa5acd766bd8dc11c/users/2123c46078684ad8b3a4045114bf817b/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:26.932672 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 822/1625] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:26.932672 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3289fe0-3760-4cdf-bbc7-ca44072b4468 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:26.937873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad5d76-1bcf-4789-ad25-a2d721cb5af7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47512038402f4e7fa5acd766bd8dc11c, user_id=2123c46078684ad8b3a4045114bf817b, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:26.940615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad5d76-1bcf-4789-ad25-a2d721cb5af7 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:26.941415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:26.952549 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3289fe0-3760-4cdf-bbc7-ca44072b4468 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:26.965517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e3f60-1939-49cf-861f-1f2931dc2568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:26.966508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e3f60-1939-49cf-861f-1f2931dc2568 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:26.966796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e3f60-1939-49cf-861f-1f2931dc2568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:26.967079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e3f60-1939-49cf-861f-1f2931dc2568 admin admin] PATH_INFO: `/v3/projects/3ff8f61d693c42ff817aea63979d4e91/users/499cf5386f5a465eb1bd3cd49bf93c1c/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:26.974063 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 804/1626] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:26 2022] PUT /identity/v3/projects/47512038402f4e7fa5acd766bd8dc11c/users/2123c46078684ad8b3a4045114bf817b/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:26.975058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad5d76-1bcf-4789-ad25-a2d721cb5af7 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:27.003127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e3f60-1939-49cf-861f-1f2931dc2568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff8f61d693c42ff817aea63979d4e91, user_id=499cf5386f5a465eb1bd3cd49bf93c1c, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:27.005898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e3f60-1939-49cf-861f-1f2931dc2568 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:27.031997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 823/1627] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:26 2022] PUT /identity/v3/projects/3ff8f61d693c42ff817aea63979d4e91/users/499cf5386f5a465eb1bd3cd49bf93c1c/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:27.035381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e3f60-1939-49cf-861f-1f2931dc2568 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:27.045650 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:27.091239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad5d76-1bcf-4789-ad25-a2d721cb5af7 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:27.200351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e3f60-1939-49cf-861f-1f2931dc2568 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:27.209567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715ce0e-0e93-49c4-a3d2-91de24766cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:27.210284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715ce0e-0e93-49c4-a3d2-91de24766cfd admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:27.210570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715ce0e-0e93-49c4-a3d2-91de24766cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:27.210850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715ce0e-0e93-49c4-a3d2-91de24766cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:27.211578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715ce0e-0e93-49c4-a3d2-91de24766cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:27.213200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715ce0e-0e93-49c4-a3d2-91de24766cfd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:27.237905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 824/1628] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:27.239641 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715ce0e-0e93-49c4-a3d2-91de24766cfd admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:27.250828 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32417731-53bb-465b-9cd1-ccaaca04616a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:27.251739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32417731-53bb-465b-9cd1-ccaaca04616a admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:27.252033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32417731-53bb-465b-9cd1-ccaaca04616a admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:27.252293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32417731-53bb-465b-9cd1-ccaaca04616a admin admin] PATH_INFO: `/v3/projects/34e1aebbb34e477d9877bd465e7d246b/users/da423103ff324bbcaa468e216cc65701/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:27.260235 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715ce0e-0e93-49c4-a3d2-91de24766cfd admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:27.274716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82700f0c-8d0e-4a4b-a998-5bdd2dc8869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:27.275395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82700f0c-8d0e-4a4b-a998-5bdd2dc8869e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:27.275687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82700f0c-8d0e-4a4b-a998-5bdd2dc8869e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:27.275957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82700f0c-8d0e-4a4b-a998-5bdd2dc8869e admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:27.276949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82700f0c-8d0e-4a4b-a998-5bdd2dc8869e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:27.278495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82700f0c-8d0e-4a4b-a998-5bdd2dc8869e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:27.283198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32417731-53bb-465b-9cd1-ccaaca04616a admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e1aebbb34e477d9877bd465e7d246b, user_id=da423103ff324bbcaa468e216cc65701, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:27.285881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32417731-53bb-465b-9cd1-ccaaca04616a admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:27.298498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 825/1629] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:27.299359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82700f0c-8d0e-4a4b-a998-5bdd2dc8869e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:27.315377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82700f0c-8d0e-4a4b-a998-5bdd2dc8869e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:27.317055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 805/1630] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:26 2022] PUT /identity/v3/projects/34e1aebbb34e477d9877bd465e7d246b/users/da423103ff324bbcaa468e216cc65701/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:27.318180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32417731-53bb-465b-9cd1-ccaaca04616a admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:27.410894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23901a-aa84-4cdf-9e2c-c01cad7208e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:27.412083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23901a-aa84-4cdf-9e2c-c01cad7208e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:27.413038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23901a-aa84-4cdf-9e2c-c01cad7208e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:27.413478 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23901a-aa84-4cdf-9e2c-c01cad7208e1 admin admin] PATH_INFO: `/v3/projects/47512038402f4e7fa5acd766bd8dc11c/users/2123c46078684ad8b3a4045114bf817b/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:27.416128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23901a-aa84-4cdf-9e2c-c01cad7208e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47512038402f4e7fa5acd766bd8dc11c, user_id=2123c46078684ad8b3a4045114bf817b, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:27.419623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23901a-aa84-4cdf-9e2c-c01cad7208e1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:27.433046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 826/1631] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:27 2022] PUT /identity/v3/projects/47512038402f4e7fa5acd766bd8dc11c/users/2123c46078684ad8b3a4045114bf817b/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:27.434631 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23901a-aa84-4cdf-9e2c-c01cad7208e1 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:27.482143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32417731-53bb-465b-9cd1-ccaaca04616a admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:27.564351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:27.568801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23901a-aa84-4cdf-9e2c-c01cad7208e1 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:27.616754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b912ea7-a242-4caf-a144-9f3ffc0db3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:27.620151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b912ea7-a242-4caf-a144-9f3ffc0db3ce admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:27.620690 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b912ea7-a242-4caf-a144-9f3ffc0db3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:27.621401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b912ea7-a242-4caf-a144-9f3ffc0db3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:27.623826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b912ea7-a242-4caf-a144-9f3ffc0db3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:27.626353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b912ea7-a242-4caf-a144-9f3ffc0db3ce admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:27.643834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 827/1632] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:27.645326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8916b0-80d3-4f11-9133-aaae042e8229 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:27.645717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8916b0-80d3-4f11-9133-aaae042e8229 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:27.646829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8916b0-80d3-4f11-9133-aaae042e8229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:27.694410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2d8916b0-80d3-4f11-9133-aaae042e8229 None None] MFA Rules not processed for user `2123c46078684ad8b3a4045114bf817b`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:27.696065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2d8916b0-80d3-4f11-9133-aaae042e8229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:27.728345 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb89979-ca72-4fed-abe9-2df48004ba2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:27.732583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb89979-ca72-4fed-abe9-2df48004ba2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:27.733503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb89979-ca72-4fed-abe9-2df48004ba2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:27.736062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb89979-ca72-4fed-abe9-2df48004ba2e admin admin] PATH_INFO: `/v3/projects/3ff8f61d693c42ff817aea63979d4e91/users/499cf5386f5a465eb1bd3cd49bf93c1c/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:27.739180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb89979-ca72-4fed-abe9-2df48004ba2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff8f61d693c42ff817aea63979d4e91, user_id=499cf5386f5a465eb1bd3cd49bf93c1c, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:27.748187 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb89979-ca72-4fed-abe9-2df48004ba2e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:27.778485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 806/1633] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:27 2022] PUT /identity/v3/projects/3ff8f61d693c42ff817aea63979d4e91/users/499cf5386f5a465eb1bd3cd49bf93c1c/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:27.779479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb89979-ca72-4fed-abe9-2df48004ba2e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:27.825584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:27.888147 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 828/1634] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:27.889249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f58c7a-f6ba-4a79-a803-21ab65800989 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:27.889622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f58c7a-f6ba-4a79-a803-21ab65800989 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:27.890473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f58c7a-f6ba-4a79-a803-21ab65800989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:27.963925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a4f58c7a-f6ba-4a79-a803-21ab65800989 None None] MFA Rules not processed for user `499cf5386f5a465eb1bd3cd49bf93c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:27.966298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a4f58c7a-f6ba-4a79-a803-21ab65800989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:27.969829 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb89979-ca72-4fed-abe9-2df48004ba2e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:27.986599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80156533-dec7-4d5a-b57a-99af2b2860f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:27.987508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80156533-dec7-4d5a-b57a-99af2b2860f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:27.987783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80156533-dec7-4d5a-b57a-99af2b2860f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:27.988058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80156533-dec7-4d5a-b57a-99af2b2860f7 admin admin] PATH_INFO: `/v3/projects/34e1aebbb34e477d9877bd465e7d246b/users/da423103ff324bbcaa468e216cc65701/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:27.990700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80156533-dec7-4d5a-b57a-99af2b2860f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e1aebbb34e477d9877bd465e7d246b, user_id=da423103ff324bbcaa468e216cc65701, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:27.997623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80156533-dec7-4d5a-b57a-99af2b2860f7 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:28.047790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 807/1635] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:27 2022] PUT /identity/v3/projects/34e1aebbb34e477d9877bd465e7d246b/users/da423103ff324bbcaa468e216cc65701/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:28.047790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80156533-dec7-4d5a-b57a-99af2b2860f7 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:28.226888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 829/1636] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:28.228526 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b398781-11b4-4124-aaba-c1ddd0ba7826 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:28.229055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b398781-11b4-4124-aaba-c1ddd0ba7826 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:28.229341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b398781-11b4-4124-aaba-c1ddd0ba7826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:28.266345 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80156533-dec7-4d5a-b57a-99af2b2860f7 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:28.276451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314959a-5d43-4be7-9417-e03c1eab1d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:28.277618 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314959a-5d43-4be7-9417-e03c1eab1d30 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:28.277898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314959a-5d43-4be7-9417-e03c1eab1d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:28.278246 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314959a-5d43-4be7-9417-e03c1eab1d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:28.302043 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4b398781-11b4-4124-aaba-c1ddd0ba7826 None None] MFA Rules not processed for user `da423103ff324bbcaa468e216cc65701`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:28.304506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4b398781-11b4-4124-aaba-c1ddd0ba7826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:28.446991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314959a-5d43-4be7-9417-e03c1eab1d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:28.447226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:28.452263 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314959a-5d43-4be7-9417-e03c1eab1d30 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:28.467469 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 808/1637] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:28.469655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314959a-5d43-4be7-9417-e03c1eab1d30 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:28.488727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314959a-5d43-4be7-9417-e03c1eab1d30 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:28.500317 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 830/1638] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:28.502998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85623b-9710-473b-b3b9-bf5805d74d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:28.503675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85623b-9710-473b-b3b9-bf5805d74d59 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:28.503961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85623b-9710-473b-b3b9-bf5805d74d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:28.504236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85623b-9710-473b-b3b9-bf5805d74d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:28.640464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:28.648080 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85623b-9710-473b-b3b9-bf5805d74d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:28.649545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85623b-9710-473b-b3b9-bf5805d74d59 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:28.700706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 809/1639] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:28.708713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b398781-11b4-4124-aaba-c1ddd0ba7826 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:28.721704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b398781-11b4-4124-aaba-c1ddd0ba7826 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:28.727191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a9e4c-3dd5-4d11-ae4f-f5bb3056f7fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:28.727998 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a9e4c-3dd5-4d11-ae4f-f5bb3056f7fb service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:28.728439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a9e4c-3dd5-4d11-ae4f-f5bb3056f7fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:28.728916 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a9e4c-3dd5-4d11-ae4f-f5bb3056f7fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:28.741398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a9e4c-3dd5-4d11-ae4f-f5bb3056f7fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:28.743417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a9e4c-3dd5-4d11-ae4f-f5bb3056f7fb service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:28.771544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 831/1640] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:28.775948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85623b-9710-473b-b3b9-bf5805d74d59 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:28.946896 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a9e4c-3dd5-4d11-ae4f-f5bb3056f7fb service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:28.960884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a9e4c-3dd5-4d11-ae4f-f5bb3056f7fb service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:28.972682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85623b-9710-473b-b3b9-bf5805d74d59 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:28.979697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb721dc-2bca-4763-b64f-cac990fce2fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:28.979697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb721dc-2bca-4763-b64f-cac990fce2fd service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:28.979697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb721dc-2bca-4763-b64f-cac990fce2fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:28.979697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb721dc-2bca-4763-b64f-cac990fce2fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:28.989587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:28.992811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:28.993099 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:28.993377 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] PATH_INFO: `/v3/users/996fbc75a3554253a9dec73ac8dc6f93` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:29.001624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=996fbc75a3554253a9dec73ac8dc6f93)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:29.003731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:29.159484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '996fbc75a3554253a9dec73ac8dc6f93'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:29.160273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '996fbc75a3554253a9dec73ac8dc6f93'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:29.161657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '996fbc75a3554253a9dec73ac8dc6f93'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:29.183299 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '996fbc75a3554253a9dec73ac8dc6f93'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:29.188541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb721dc-2bca-4763-b64f-cac990fce2fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:29.190055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb721dc-2bca-4763-b64f-cac990fce2fd service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:29.214350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '996fbc75a3554253a9dec73ac8dc6f93'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:29.218563 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 832/1641] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:39:29.254669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 810/1642] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:39:28 2022] DELETE /identity/v3/users/996fbc75a3554253a9dec73ac8dc6f93 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:29.257014 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59933ad-d1c0-4957-8aee-cfd0d7a80af6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:29.257518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59933ad-d1c0-4957-8aee-cfd0d7a80af6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:29.257897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59933ad-d1c0-4957-8aee-cfd0d7a80af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:29.302920 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d59933ad-d1c0-4957-8aee-cfd0d7a80af6 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:29.310356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d59933ad-d1c0-4957-8aee-cfd0d7a80af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:29.470954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 833/1643] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:29.547659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:29.554931 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:29.597741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8d07-5249-4bbc-af69-acd7b15f6791 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:29.598130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8d07-5249-4bbc-af69-acd7b15f6791 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:29.598476 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8d07-5249-4bbc-af69-acd7b15f6791 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:29.599926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 834/1644] 10.222.0.60 () {54 vars in 962 bytes} [Mon Jul 25 12:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:39:29.628770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87c969-d0aa-4774-8e91-bec48dab0e60 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:29.629188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87c969-d0aa-4774-8e91-bec48dab0e60 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:29.629563 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87c969-d0aa-4774-8e91-bec48dab0e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:29.704777 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc244a-a909-426a-89e8-7cbe6ca5b789 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:29.709714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc162217-f4a8-4c2b-bc92-2d230c9d177d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:29.710534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc162217-f4a8-4c2b-bc92-2d230c9d177d service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:29.710957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc162217-f4a8-4c2b-bc92-2d230c9d177d service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:29.711266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc162217-f4a8-4c2b-bc92-2d230c9d177d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:29.737805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc162217-f4a8-4c2b-bc92-2d230c9d177d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:29.740816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc162217-f4a8-4c2b-bc92-2d230c9d177d service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:29.749894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9c87c969-d0aa-4774-8e91-bec48dab0e60 None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:29.750921 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9c87c969-d0aa-4774-8e91-bec48dab0e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:29.771181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 811/1645] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:29.792538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc162217-f4a8-4c2b-bc92-2d230c9d177d service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:29.794410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fc162217-f4a8-4c2b-bc92-2d230c9d177d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:29.816892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc162217-f4a8-4c2b-bc92-2d230c9d177d service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:29.831077 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70863c-8a1e-417f-8371-c1f35e392e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:29.831970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70863c-8a1e-417f-8371-c1f35e392e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:29.832255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70863c-8a1e-417f-8371-c1f35e392e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:29.832534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70863c-8a1e-417f-8371-c1f35e392e90 admin admin] PATH_INFO: `/v3/users/cc58762391d1495187ad44a7f9c0d64f` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:29.840545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70863c-8a1e-417f-8371-c1f35e392e90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc58762391d1495187ad44a7f9c0d64f)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:29.842574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70863c-8a1e-417f-8371-c1f35e392e90 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:29.867423 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:29.945471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2d70863c-8a1e-417f-8371-c1f35e392e90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc58762391d1495187ad44a7f9c0d64f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:29.946125 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2d70863c-8a1e-417f-8371-c1f35e392e90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc58762391d1495187ad44a7f9c0d64f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:29.954465 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2d70863c-8a1e-417f-8371-c1f35e392e90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc58762391d1495187ad44a7f9c0d64f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:30.014152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2d70863c-8a1e-417f-8371-c1f35e392e90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc58762391d1495187ad44a7f9c0d64f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:30.019200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:30.020238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2d70863c-8a1e-417f-8371-c1f35e392e90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc58762391d1495187ad44a7f9c0d64f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:30.023064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 835/1646] 10.222.0.60 () {58 vars in 1059 bytes} [Mon Jul 25 12:39:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:39:30.043448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2fe5a-5af8-43b1-bc91-c42b164d4bdf None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:30.043705 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2fe5a-5af8-43b1-bc91-c42b164d4bdf None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:30.043982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2fe5a-5af8-43b1-bc91-c42b164d4bdf None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:30.045844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 836/1647] 10.222.0.60 () {54 vars in 912 bytes} [Mon Jul 25 12:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:39:30.069939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d7a58-4b54-464a-bc9f-2bee1a2cda96 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:30.070320 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d7a58-4b54-464a-bc9f-2bee1a2cda96 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:30.070539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d7a58-4b54-464a-bc9f-2bee1a2cda96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:30.122915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 812/1648] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:39:29 2022] DELETE /identity/v3/users/cc58762391d1495187ad44a7f9c0d64f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:39:30.127808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5c5c9-5fa5-412d-8b1f-6ac51160e4a8 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:30.128080 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5c5c9-5fa5-412d-8b1f-6ac51160e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:30.130383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5c5c9-5fa5-412d-8b1f-6ac51160e4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:30.135983 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e90d7a58-4b54-464a-bc9f-2bee1a2cda96 None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:30.137241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e90d7a58-4b54-464a-bc9f-2bee1a2cda96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:30.165945 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-4fb5c5c9-5fa5-412d-8b1f-6ac51160e4a8 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:30.168691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4fb5c5c9-5fa5-412d-8b1f-6ac51160e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:30.289707 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 837/1649] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:39:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:39:30.310832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d7a58-4b54-464a-bc9f-2bee1a2cda96 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:30.315239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e90d7a58-4b54-464a-bc9f-2bee1a2cda96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:30.331181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 813/1650] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:30.418930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5c5c9-5fa5-412d-8b1f-6ac51160e4a8 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:30.420680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4fb5c5c9-5fa5-412d-8b1f-6ac51160e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:30.441271 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d7a58-4b54-464a-bc9f-2bee1a2cda96 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:30.445719 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28637114-e88e-41a1-8d40-cc7d49834fea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:30.447032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28637114-e88e-41a1-8d40-cc7d49834fea service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:30.447503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28637114-e88e-41a1-8d40-cc7d49834fea service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:30.447640 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28637114-e88e-41a1-8d40-cc7d49834fea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:30.475856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28637114-e88e-41a1-8d40-cc7d49834fea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:30.489218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28637114-e88e-41a1-8d40-cc7d49834fea service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:30.508117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 838/1651] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:30.647966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5c5c9-5fa5-412d-8b1f-6ac51160e4a8 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:30.663195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17516352-26e4-4ef4-9f8c-355d19c03b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:30.663877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17516352-26e4-4ef4-9f8c-355d19c03b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:30.664150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17516352-26e4-4ef4-9f8c-355d19c03b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:30.664424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17516352-26e4-4ef4-9f8c-355d19c03b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:30.695740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17516352-26e4-4ef4-9f8c-355d19c03b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:30.703141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17516352-26e4-4ef4-9f8c-355d19c03b4a service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:30.748397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 814/1652] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:39:31.624797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28637114-e88e-41a1-8d40-cc7d49834fea service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:31.627472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-28637114-e88e-41a1-8d40-cc7d49834fea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:31.657886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28637114-e88e-41a1-8d40-cc7d49834fea service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:31.672714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7791d-0989-4fea-a3b8-a33fce39ee20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:31.674405 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7791d-0989-4fea-a3b8-a33fce39ee20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:31.674925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7791d-0989-4fea-a3b8-a33fce39ee20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:31.675080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7791d-0989-4fea-a3b8-a33fce39ee20 admin admin] PATH_INFO: `/v3/projects/43179fa28eb0451ca665d1d6b0e70b73` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:31.681669 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7791d-0989-4fea-a3b8-a33fce39ee20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43179fa28eb0451ca665d1d6b0e70b73)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:31.684202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7791d-0989-4fea-a3b8-a33fce39ee20 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:31.789524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a8f7791d-0989-4fea-a3b8-a33fce39ee20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43179fa28eb0451ca665d1d6b0e70b73', 'request_id': 'req-a8f7791d-0989-4fea-a3b8-a33fce39ee20'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:31.808183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a8f7791d-0989-4fea-a3b8-a33fce39ee20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43179fa28eb0451ca665d1d6b0e70b73', 'request_id': 'req-a8f7791d-0989-4fea-a3b8-a33fce39ee20'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:31.824824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a8f7791d-0989-4fea-a3b8-a33fce39ee20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43179fa28eb0451ca665d1d6b0e70b73', 'request_id': 'req-a8f7791d-0989-4fea-a3b8-a33fce39ee20'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:31.847663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a8f7791d-0989-4fea-a3b8-a33fce39ee20 admin admin] The token cache is being invalidate because project 43179fa28eb0451ca665d1d6b0e70b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:39:31.848285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a8f7791d-0989-4fea-a3b8-a33fce39ee20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:31.849881 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a8f7791d-0989-4fea-a3b8-a33fce39ee20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:31.852661 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 839/1653] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:39:31 2022] DELETE /identity/v3/projects/43179fa28eb0451ca665d1d6b0e70b73 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:31.856418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17516352-26e4-4ef4-9f8c-355d19c03b4a service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:31.872814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17516352-26e4-4ef4-9f8c-355d19c03b4a service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:31.973478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:32.024616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:32.026111 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:32.026605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:32.027131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] PATH_INFO: `/v3/users/499cf5386f5a465eb1bd3cd49bf93c1c` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:32.028734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=499cf5386f5a465eb1bd3cd49bf93c1c)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:32.032037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:32.090909 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '499cf5386f5a465eb1bd3cd49bf93c1c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:32.091714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '499cf5386f5a465eb1bd3cd49bf93c1c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:32.092418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '499cf5386f5a465eb1bd3cd49bf93c1c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:32.106348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '499cf5386f5a465eb1bd3cd49bf93c1c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:32.116984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '499cf5386f5a465eb1bd3cd49bf93c1c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:32.149586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 815/1654] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:39:31 2022] DELETE /identity/v3/users/499cf5386f5a465eb1bd3cd49bf93c1c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:32.520112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851548fb-7c22-4b67-ac67-48e64a4d57e1 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:32.520402 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851548fb-7c22-4b67-ac67-48e64a4d57e1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:32.520685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851548fb-7c22-4b67-ac67-48e64a4d57e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:32.564037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:32.583483 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-851548fb-7c22-4b67-ac67-48e64a4d57e1 None None] MFA Rules not processed for user `e56531920fc548a3a37f63ca0d1e5d82`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:32.586023 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-851548fb-7c22-4b67-ac67-48e64a4d57e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:32.772803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:32.774589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:32.809637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 840/1655] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:32.814380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84713ef7-1ad2-4d4e-9bad-b0f3691cd0f6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:32.814773 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84713ef7-1ad2-4d4e-9bad-b0f3691cd0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:32.815709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84713ef7-1ad2-4d4e-9bad-b0f3691cd0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:32.876486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-84713ef7-1ad2-4d4e-9bad-b0f3691cd0f6 None None] MFA Rules not processed for user `e56531920fc548a3a37f63ca0d1e5d82`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:32.878348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-84713ef7-1ad2-4d4e-9bad-b0f3691cd0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:32.898212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 841/1656] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:32.910990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84713ef7-1ad2-4d4e-9bad-b0f3691cd0f6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:32.912680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-84713ef7-1ad2-4d4e-9bad-b0f3691cd0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:32.987549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386a2cf-8421-47ff-9f7a-fc4891e38fd9 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:33.005794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:33.006726 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:33.007049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:33.007328 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] PATH_INFO: `/v3/projects/7dfb4873fce0400b88e139274d392e58` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:33.010713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dfb4873fce0400b88e139274d392e58)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:33.011753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: WARNING py.warnings [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:39:33.011753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: warnings.warn(msg) Jul 25 12:39:33.011753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: Jul 25 12:39:33.013018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:33.131118 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dfb4873fce0400b88e139274d392e58', 'request_id': 'req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.150000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dfb4873fce0400b88e139274d392e58', 'request_id': 'req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.152008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:33.161287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dfb4873fce0400b88e139274d392e58', 'request_id': 'req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.192845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] The token cache is being invalidate because project 7dfb4873fce0400b88e139274d392e58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:39:33.193336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.194324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.197125 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 816/1657] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:39:32 2022] DELETE /identity/v3/projects/7dfb4873fce0400b88e139274d392e58 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:39:33.198222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:33.201729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:33.212249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84713ef7-1ad2-4d4e-9bad-b0f3691cd0f6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:33.344379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af42e8d-3f2d-4a1d-ba22-5fbd444377f8 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:33.356512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:33.357397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:33.358805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:33.358805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2 admin admin] PATH_INFO: `/v3/projects/3ff8f61d693c42ff817aea63979d4e91` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:33.361291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651122a1-6139-4d2a-87b7-a9eef76c3330 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:33.361291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651122a1-6139-4d2a-87b7-a9eef76c3330 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:33.361509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ff8f61d693c42ff817aea63979d4e91)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:33.362051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651122a1-6139-4d2a-87b7-a9eef76c3330 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:33.363567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651122a1-6139-4d2a-87b7-a9eef76c3330 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:33.365535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:33.367969 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-651122a1-6139-4d2a-87b7-a9eef76c3330 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:33.447868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:33.471869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ff8f61d693c42ff817aea63979d4e91', 'request_id': 'req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.493721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ff8f61d693c42ff817aea63979d4e91', 'request_id': 'req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.504869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651122a1-6139-4d2a-87b7-a9eef76c3330 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:33.506666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651122a1-6139-4d2a-87b7-a9eef76c3330 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:33.515839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ff8f61d693c42ff817aea63979d4e91', 'request_id': 'req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.517688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 842/1658] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:39:33.519091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabbc6e-16af-45c3-a4ff-4ff4b164bdd2 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:33.519373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabbc6e-16af-45c3-a4ff-4ff4b164bdd2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:33.519644 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabbc6e-16af-45c3-a4ff-4ff4b164bdd2 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:33.521686 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 843/1659] 10.222.0.60 () {54 vars in 962 bytes} [Mon Jul 25 12:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:39:33.522533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabbc6e-16af-45c3-a4ff-4ff4b164bdd2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:33.524681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2dabbc6e-16af-45c3-a4ff-4ff4b164bdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:33.565937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2 admin admin] The token cache is being invalidate because project 3ff8f61d693c42ff817aea63979d4e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:39:33.566243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.567193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-3686c3e2-d38f-4d56-9084-32d9c7bbb0b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.572357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 817/1660] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:39:33 2022] DELETE /identity/v3/projects/3ff8f61d693c42ff817aea63979d4e91 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:33.572357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60266c2-057c-4540-aa43-9bd2d1c4cb4d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:33.572544 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60266c2-057c-4540-aa43-9bd2d1c4cb4d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:33.572775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60266c2-057c-4540-aa43-9bd2d1c4cb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:33.621595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-d60266c2-057c-4540-aa43-9bd2d1c4cb4d None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:33.625002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d60266c2-057c-4540-aa43-9bd2d1c4cb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:33.704874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabbc6e-16af-45c3-a4ff-4ff4b164bdd2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:33.718950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:33.720104 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:33.720484 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:33.720834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] PATH_INFO: `/v3/users/da423103ff324bbcaa468e216cc65701` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:33.724124 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=da423103ff324bbcaa468e216cc65701)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:33.726161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: WARNING py.warnings [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:39:33.726161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: warnings.warn(msg) Jul 25 12:39:33.726161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: Jul 25 12:39:33.729945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:33.783800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 818/1661] 10.222.0.60 () {58 vars in 1059 bytes} [Mon Jul 25 12:39:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:33.789489 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398a13b-f792-45f4-80c2-36004786be7a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:33.789785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398a13b-f792-45f4-80c2-36004786be7a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:33.790235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398a13b-f792-45f4-80c2-36004786be7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:33.813871 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:33.835800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-3398a13b-f792-45f4-80c2-36004786be7a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:33.840880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-3398a13b-f792-45f4-80c2-36004786be7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:33.863216 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 819/1662] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:33.869283 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9bcfd-a50a-42b9-988c-948fd5e803f6 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:33.869652 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9bcfd-a50a-42b9-988c-948fd5e803f6 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:33.869999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9bcfd-a50a-42b9-988c-948fd5e803f6 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:33.879987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 820/1663] 10.222.0.60 () {54 vars in 912 bytes} [Mon Jul 25 12:39:33 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:39:33.882821 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7c918-7eba-4fc5-9ce7-ea7f66a4bd99 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:33.883144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7c918-7eba-4fc5-9ce7-ea7f66a4bd99 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:33.883429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7c918-7eba-4fc5-9ce7-ea7f66a4bd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:33.883768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da423103ff324bbcaa468e216cc65701'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.897229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da423103ff324bbcaa468e216cc65701'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.899320 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da423103ff324bbcaa468e216cc65701'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.926332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da423103ff324bbcaa468e216cc65701'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.928917 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da423103ff324bbcaa468e216cc65701'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:33.938561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-37f7c918-7eba-4fc5-9ce7-ea7f66a4bd99 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:33.940052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-37f7c918-7eba-4fc5-9ce7-ea7f66a4bd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:33.956331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 821/1664] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:33.958582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0133e3-6a79-4926-8ed7-9e8730183983 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:33.959430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0133e3-6a79-4926-8ed7-9e8730183983 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:33.961759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0133e3-6a79-4926-8ed7-9e8730183983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:33.974489 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 844/1665] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:39:33 2022] DELETE /identity/v3/users/da423103ff324bbcaa468e216cc65701 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:33.975481 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:34.011967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-0f0133e3-6a79-4926-8ed7-9e8730183983 None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:34.013208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0f0133e3-6a79-4926-8ed7-9e8730183983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:34.054587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:34.087977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2cac6-824d-47a8-b26d-4bab13cdcd9c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:34.106642 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39effcd-71e0-46ed-a3d7-b95e7e73d33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:34.107364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39effcd-71e0-46ed-a3d7-b95e7e73d33b admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:34.107654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39effcd-71e0-46ed-a3d7-b95e7e73d33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:34.110081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39effcd-71e0-46ed-a3d7-b95e7e73d33b admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:34.113244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39effcd-71e0-46ed-a3d7-b95e7e73d33b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:34.113485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39effcd-71e0-46ed-a3d7-b95e7e73d33b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:34.119181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39effcd-71e0-46ed-a3d7-b95e7e73d33b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:34.159044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 845/1666] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:39:34.161380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39effcd-71e0-46ed-a3d7-b95e7e73d33b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:34.205372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39effcd-71e0-46ed-a3d7-b95e7e73d33b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:34.208194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 822/1667] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:39:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:34.212394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183a33d-aca3-45eb-a8fe-c3642e9532ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:34.213204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183a33d-aca3-45eb-a8fe-c3642e9532ca admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:34.213498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183a33d-aca3-45eb-a8fe-c3642e9532ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:34.213782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183a33d-aca3-45eb-a8fe-c3642e9532ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:34.214663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183a33d-aca3-45eb-a8fe-c3642e9532ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:34.215665 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0133e3-6a79-4926-8ed7-9e8730183983 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:34.219338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0f0133e3-6a79-4926-8ed7-9e8730183983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:34.219466 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183a33d-aca3-45eb-a8fe-c3642e9532ca admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:34.304949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 846/1668] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:34 2022] POST /identity/v3/projects => generated 391 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:34.306412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183a33d-aca3-45eb-a8fe-c3642e9532ca admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:34.359307 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0133e3-6a79-4926-8ed7-9e8730183983 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:34.472165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183a33d-aca3-45eb-a8fe-c3642e9532ca admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:34.488436 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0ecc6-55f7-442b-9efc-d45b99d61e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:34.489141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ecc6-55f7-442b-9efc-d45b99d61e98 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:34.489427 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ecc6-55f7-442b-9efc-d45b99d61e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:34.489713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ecc6-55f7-442b-9efc-d45b99d61e98 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:34.490470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0ecc6-55f7-442b-9efc-d45b99d61e98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:34.494480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0ecc6-55f7-442b-9efc-d45b99d61e98 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:34.503620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58729c5-acd7-432b-8e40-997ee8bbbb30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:34.504616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58729c5-acd7-432b-8e40-997ee8bbbb30 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:34.504923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58729c5-acd7-432b-8e40-997ee8bbbb30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:34.505188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58729c5-acd7-432b-8e40-997ee8bbbb30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:34.558950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 847/1669] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:34 2022] POST /identity/v3/users => generated 296 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:34.560338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0ecc6-55f7-442b-9efc-d45b99d61e98 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:34.577851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0ecc6-55f7-442b-9efc-d45b99d61e98 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:34.592807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aafb18-4fcd-478b-903b-c9ef0f659240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:34.593503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aafb18-4fcd-478b-903b-c9ef0f659240 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:34.593788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aafb18-4fcd-478b-903b-c9ef0f659240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:34.594095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aafb18-4fcd-478b-903b-c9ef0f659240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:34.594841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aafb18-4fcd-478b-903b-c9ef0f659240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:34.596699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aafb18-4fcd-478b-903b-c9ef0f659240 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:34.619776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:34.626863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 848/1670] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:34.630670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aafb18-4fcd-478b-903b-c9ef0f659240 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:34.655478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58729c5-acd7-432b-8e40-997ee8bbbb30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:34.657111 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58729c5-acd7-432b-8e40-997ee8bbbb30 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:34.669140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aafb18-4fcd-478b-903b-c9ef0f659240 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:34.675942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec126a6-1a55-4cbc-9ee4-19ac8a3be507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:34.676967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec126a6-1a55-4cbc-9ee4-19ac8a3be507 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:34.677452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec126a6-1a55-4cbc-9ee4-19ac8a3be507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:34.678103 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec126a6-1a55-4cbc-9ee4-19ac8a3be507 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:34.679117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec126a6-1a55-4cbc-9ee4-19ac8a3be507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:34.682158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec126a6-1a55-4cbc-9ee4-19ac8a3be507 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:34.685676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 823/1671] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:39:34.704826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 849/1672] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:34.712394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58729c5-acd7-432b-8e40-997ee8bbbb30 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:34.727840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58729c5-acd7-432b-8e40-997ee8bbbb30 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:34.751523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61280a-3a7f-478f-b5eb-560686b75ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:34.752443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61280a-3a7f-478f-b5eb-560686b75ced admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:34.752729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61280a-3a7f-478f-b5eb-560686b75ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:34.753010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61280a-3a7f-478f-b5eb-560686b75ced admin admin] PATH_INFO: `/v3/projects/8e785d3c68c94bf48a10c8e556d6b993/users/b0008e054713493aa956f632dfba0d4f/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:34.802458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61280a-3a7f-478f-b5eb-560686b75ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e785d3c68c94bf48a10c8e556d6b993, user_id=b0008e054713493aa956f632dfba0d4f, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:34.805049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61280a-3a7f-478f-b5eb-560686b75ced admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:34.850716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 824/1673] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:34 2022] PUT /identity/v3/projects/8e785d3c68c94bf48a10c8e556d6b993/users/b0008e054713493aa956f632dfba0d4f/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:39:34.863548 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609f214-d11c-4cb6-870b-e237510a6a19 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:34.866412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609f214-d11c-4cb6-870b-e237510a6a19 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:34.866705 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609f214-d11c-4cb6-870b-e237510a6a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:34.981265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c609f214-d11c-4cb6-870b-e237510a6a19 None None] MFA Rules not processed for user `b0008e054713493aa956f632dfba0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:34.982584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c609f214-d11c-4cb6-870b-e237510a6a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:35.237283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 850/1674] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:35.238769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61280a-3a7f-478f-b5eb-560686b75ced admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:35.403200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61280a-3a7f-478f-b5eb-560686b75ced admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:35.425624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e0741-8e33-48af-8f41-036d9dc01b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:35.426888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e0741-8e33-48af-8f41-036d9dc01b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:35.427161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e0741-8e33-48af-8f41-036d9dc01b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:35.427442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e0741-8e33-48af-8f41-036d9dc01b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:35.428160 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e0741-8e33-48af-8f41-036d9dc01b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:35.432460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e0741-8e33-48af-8f41-036d9dc01b32 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:35.525488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609f214-d11c-4cb6-870b-e237510a6a19 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:35.535460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 825/1675] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:35 2022] POST /identity/v3/projects => generated 391 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:35.548053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609f214-d11c-4cb6-870b-e237510a6a19 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:35.597015 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:35.797820 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d2a81-5e23-4cab-a92f-668fe9e882ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:35.798795 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d2a81-5e23-4cab-a92f-668fe9e882ec admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:35.799030 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d2a81-5e23-4cab-a92f-668fe9e882ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:35.799313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d2a81-5e23-4cab-a92f-668fe9e882ec admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:35.800638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d2a81-5e23-4cab-a92f-668fe9e882ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:35.806357 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d2a81-5e23-4cab-a92f-668fe9e882ec admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:35.879884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 851/1676] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:35 2022] POST /identity/v3/users => generated 302 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:39:35.881398 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e0741-8e33-48af-8f41-036d9dc01b32 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:35.900871 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e0741-8e33-48af-8f41-036d9dc01b32 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:35.927752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d6088-4c98-4799-b7bf-671c8e436f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:35.928456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d6088-4c98-4799-b7bf-671c8e436f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:35.928740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d6088-4c98-4799-b7bf-671c8e436f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:35.929018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d6088-4c98-4799-b7bf-671c8e436f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:35.935993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d6088-4c98-4799-b7bf-671c8e436f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:35.937990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d6088-4c98-4799-b7bf-671c8e436f22 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:35.960805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70c1ae-0b2f-46cf-846a-417f7d0f3f22 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:35.961200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70c1ae-0b2f-46cf-846a-417f7d0f3f22 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:35.961736 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70c1ae-0b2f-46cf-846a-417f7d0f3f22 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:35.970438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 852/1677] 10.222.0.60 () {54 vars in 912 bytes} [Mon Jul 25 12:39:35 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:39:35.970605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 826/1678] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:35.971819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70c1ae-0b2f-46cf-846a-417f7d0f3f22 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:35.977899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb3c72-7086-4117-9a30-661d1d9c7dff None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:35.978229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb3c72-7086-4117-9a30-661d1d9c7dff None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:35.978512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb3c72-7086-4117-9a30-661d1d9c7dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:36.041662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70c1ae-0b2f-46cf-846a-417f7d0f3f22 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:36.063380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539b6d0-cf13-4b75-ade8-8427e6c5fbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:36.064891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539b6d0-cf13-4b75-ade8-8427e6c5fbeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:36.065368 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539b6d0-cf13-4b75-ade8-8427e6c5fbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:36.065775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539b6d0-cf13-4b75-ade8-8427e6c5fbeb admin admin] PATH_INFO: `/v3/projects/9856cf40febc4e79b9e5e0a00cf27be0/users/a357371ffbf44038b660bc924b2fef27/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:36.113863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539b6d0-cf13-4b75-ade8-8427e6c5fbeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9856cf40febc4e79b9e5e0a00cf27be0, user_id=a357371ffbf44038b660bc924b2fef27, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:36.118535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b9eb3c72-7086-4117-9a30-661d1d9c7dff None None] MFA Rules not processed for user `375d321d6de2494794254e50151c02c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:36.119996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b9eb3c72-7086-4117-9a30-661d1d9c7dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:36.120997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539b6d0-cf13-4b75-ade8-8427e6c5fbeb admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:36.167604 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 853/1679] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:35 2022] PUT /identity/v3/projects/9856cf40febc4e79b9e5e0a00cf27be0/users/a357371ffbf44038b660bc924b2fef27/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:36.171819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539b6d0-cf13-4b75-ade8-8427e6c5fbeb admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:36.307783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 827/1680] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:39:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:36.406540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539b6d0-cf13-4b75-ade8-8427e6c5fbeb admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:36.426846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325dfa29-4deb-4676-944c-c3e6f5e02198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:36.427782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dfa29-4deb-4676-944c-c3e6f5e02198 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:36.428865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dfa29-4deb-4676-944c-c3e6f5e02198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:36.429529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dfa29-4deb-4676-944c-c3e6f5e02198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:36.444764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325dfa29-4deb-4676-944c-c3e6f5e02198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:36.450617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325dfa29-4deb-4676-944c-c3e6f5e02198 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:36.495394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 854/1681] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:39:36.500730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb3c72-7086-4117-9a30-661d1d9c7dff None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:36.526062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb3c72-7086-4117-9a30-661d1d9c7dff None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:36.551752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a817f-8230-46ef-ac68-fc0c08f567a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:36.552681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a817f-8230-46ef-ac68-fc0c08f567a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:36.552966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a817f-8230-46ef-ac68-fc0c08f567a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:36.553237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a817f-8230-46ef-ac68-fc0c08f567a9 admin admin] PATH_INFO: `/v3/projects/9856cf40febc4e79b9e5e0a00cf27be0/users/a357371ffbf44038b660bc924b2fef27/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:36.558120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a817f-8230-46ef-ac68-fc0c08f567a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9856cf40febc4e79b9e5e0a00cf27be0, user_id=a357371ffbf44038b660bc924b2fef27, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:36.562078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a817f-8230-46ef-ac68-fc0c08f567a9 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:36.606812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 828/1682] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:36 2022] PUT /identity/v3/projects/9856cf40febc4e79b9e5e0a00cf27be0/users/a357371ffbf44038b660bc924b2fef27/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:36.622246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111d99a-52aa-401c-81ff-94f577fe5fe4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:36.622528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111d99a-52aa-401c-81ff-94f577fe5fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:36.623181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111d99a-52aa-401c-81ff-94f577fe5fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:36.718044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0111d99a-52aa-401c-81ff-94f577fe5fe4 None None] MFA Rules not processed for user `a357371ffbf44038b660bc924b2fef27`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:36.727442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0111d99a-52aa-401c-81ff-94f577fe5fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:36.764446 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a817f-8230-46ef-ac68-fc0c08f567a9 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:36.766379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-422a817f-8230-46ef-ac68-fc0c08f567a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:36.827605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:36.981075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a817f-8230-46ef-ac68-fc0c08f567a9 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:36.990088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c703465-be46-4cad-9f5e-5a3934a27bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:36.990760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c703465-be46-4cad-9f5e-5a3934a27bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:36.991043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c703465-be46-4cad-9f5e-5a3934a27bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:36.991318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c703465-be46-4cad-9f5e-5a3934a27bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:37.007569 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c703465-be46-4cad-9f5e-5a3934a27bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:37.013504 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c703465-be46-4cad-9f5e-5a3934a27bf8 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:37.035939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 829/1683] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:37.077006 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c703465-be46-4cad-9f5e-5a3934a27bf8 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:37.078456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 855/1684] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:37.136373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95db9e-f9b9-4878-9c40-79b3b5ce4e79 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:37.136663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95db9e-f9b9-4878-9c40-79b3b5ce4e79 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:37.136943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95db9e-f9b9-4878-9c40-79b3b5ce4e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:37.152981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:37.231064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2b95db9e-f9b9-4878-9c40-79b3b5ce4e79 None None] MFA Rules not processed for user `1bc5799ac34349518854829ec2c259f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:37.232356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2b95db9e-f9b9-4878-9c40-79b3b5ce4e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:37.266739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c703465-be46-4cad-9f5e-5a3934a27bf8 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:37.280542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640142e-76db-4724-9b84-187d940b33b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:37.281676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640142e-76db-4724-9b84-187d940b33b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:37.282162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640142e-76db-4724-9b84-187d940b33b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:37.282614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640142e-76db-4724-9b84-187d940b33b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:37.326640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640142e-76db-4724-9b84-187d940b33b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:37.328501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640142e-76db-4724-9b84-187d940b33b1 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:37.362710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 830/1685] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:37.363885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640142e-76db-4724-9b84-187d940b33b1 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:37.368796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e640142e-76db-4724-9b84-187d940b33b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:37.397502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 856/1686] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:37.413936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95db9e-f9b9-4878-9c40-79b3b5ce4e79 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:37.415442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2b95db9e-f9b9-4878-9c40-79b3b5ce4e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:37.528055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640142e-76db-4724-9b84-187d940b33b1 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:37.536890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:37.538501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:37.538934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:37.539350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] PATH_INFO: `/v3/projects/34e1aebbb34e477d9877bd465e7d246b` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:37.547047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] RBAC: Authorizing `identity:delete_project(project_id=34e1aebbb34e477d9877bd465e7d246b)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:37.553652 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:37.614870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95db9e-f9b9-4878-9c40-79b3b5ce4e79 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:37.625268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34e1aebbb34e477d9877bd465e7d246b', 'request_id': 'req-6d745cf5-755b-40cd-a97c-23f17f26378a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:37.636840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34e1aebbb34e477d9877bd465e7d246b', 'request_id': 'req-6d745cf5-755b-40cd-a97c-23f17f26378a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:37.643164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:37.644993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34e1aebbb34e477d9877bd465e7d246b', 'request_id': 'req-6d745cf5-755b-40cd-a97c-23f17f26378a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:37.677564 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] The token cache is being invalidate because project 34e1aebbb34e477d9877bd465e7d246b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:39:37.677858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:37.679657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:37.681972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 831/1687] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:39:37 2022] DELETE /identity/v3/projects/34e1aebbb34e477d9877bd465e7d246b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:37.683215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:37.686107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:37.792655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90717520-b1c5-4ce4-8373-d6efcad4a8b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:37.793438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717520-b1c5-4ce4-8373-d6efcad4a8b8 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:37.793610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717520-b1c5-4ce4-8373-d6efcad4a8b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:37.793887 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717520-b1c5-4ce4-8373-d6efcad4a8b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:37.799571 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-90717520-b1c5-4ce4-8373-d6efcad4a8b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:37.829666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745cf5-755b-40cd-a97c-23f17f26378a admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:37.838533 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa1bfd-defa-4934-9a1a-ae986d06db59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:37.839522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa1bfd-defa-4934-9a1a-ae986d06db59 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:37.839811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa1bfd-defa-4934-9a1a-ae986d06db59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:37.840184 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa1bfd-defa-4934-9a1a-ae986d06db59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:37.844359 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e0aa1bfd-defa-4934-9a1a-ae986d06db59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:37.917498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:37.985687 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa1bfd-defa-4934-9a1a-ae986d06db59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:37.987631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa1bfd-defa-4934-9a1a-ae986d06db59 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:38.011834 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 832/1688] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:38.013527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb525d4-e17b-4841-94af-b5b86a142322 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:38.013941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb525d4-e17b-4841-94af-b5b86a142322 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:38.016303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb525d4-e17b-4841-94af-b5b86a142322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:38.021572 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90717520-b1c5-4ce4-8373-d6efcad4a8b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:38.025622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90717520-b1c5-4ce4-8373-d6efcad4a8b8 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:38.049256 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-2fb525d4-e17b-4841-94af-b5b86a142322 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:38.050387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 857/1689] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:38.050974 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2fb525d4-e17b-4841-94af-b5b86a142322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:38.152323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 833/1690] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:38.155097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4550755-e56c-4514-af7e-2068d1b83344 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:38.155650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4550755-e56c-4514-af7e-2068d1b83344 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:38.155970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4550755-e56c-4514-af7e-2068d1b83344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:38.211754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d4550755-e56c-4514-af7e-2068d1b83344 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:38.213013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d4550755-e56c-4514-af7e-2068d1b83344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:38.238687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 858/1691] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:38.240061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb525d4-e17b-4841-94af-b5b86a142322 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:38.251885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb525d4-e17b-4841-94af-b5b86a142322 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:38.266705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb719b1b-0999-41e9-9d72-d15388c929dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:38.267950 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb719b1b-0999-41e9-9d72-d15388c929dd admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:38.268231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb719b1b-0999-41e9-9d72-d15388c929dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:38.269946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb719b1b-0999-41e9-9d72-d15388c929dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:38.270739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb719b1b-0999-41e9-9d72-d15388c929dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:38.272135 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb719b1b-0999-41e9-9d72-d15388c929dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:38.273737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb719b1b-0999-41e9-9d72-d15388c929dd admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:38.305955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 834/1692] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:39:38.310790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4550755-e56c-4514-af7e-2068d1b83344 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:38.354499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4550755-e56c-4514-af7e-2068d1b83344 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:38.370166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fb3f8-5c87-4a99-b377-0ad3019af62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:38.371200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fb3f8-5c87-4a99-b377-0ad3019af62b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:38.371484 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fb3f8-5c87-4a99-b377-0ad3019af62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:38.371808 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fb3f8-5c87-4a99-b377-0ad3019af62b admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:38.373454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fb3f8-5c87-4a99-b377-0ad3019af62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:38.378754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fb3f8-5c87-4a99-b377-0ad3019af62b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:38.407559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be45a99-f5d7-4e7c-bcd4-3033044e4c46 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:38.408002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be45a99-f5d7-4e7c-bcd4-3033044e4c46 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:38.408605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be45a99-f5d7-4e7c-bcd4-3033044e4c46 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:38.409497 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 835/1693] 10.222.0.60 () {54 vars in 961 bytes} [Mon Jul 25 12:39:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:39:38.422594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a74c2-6227-4f7b-b18f-7e87133de381 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:38.423094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a74c2-6227-4f7b-b18f-7e87133de381 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:38.423309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a74c2-6227-4f7b-b18f-7e87133de381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:38.495446 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-057a74c2-6227-4f7b-b18f-7e87133de381 None None] MFA Rules not processed for user `81037f61e01e4a93982503720f7404e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:38.496765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-057a74c2-6227-4f7b-b18f-7e87133de381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:38.504912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 859/1694] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:38 2022] POST /identity/v3/projects => generated 357 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:39:38.506291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fb3f8-5c87-4a99-b377-0ad3019af62b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:38.573565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:38.659157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fb3f8-5c87-4a99-b377-0ad3019af62b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:38.691772 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 836/1695] 10.222.0.60 () {58 vars in 1058 bytes} [Mon Jul 25 12:39:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 25 12:39:38.703816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34c65b-d6a2-4e11-8fa4-bff310c55346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:38.703978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a74c2-6227-4f7b-b18f-7e87133de381 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:38.704100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34c65b-d6a2-4e11-8fa4-bff310c55346 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:38.704100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34c65b-d6a2-4e11-8fa4-bff310c55346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:38.704100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34c65b-d6a2-4e11-8fa4-bff310c55346 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:38.704647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34c65b-d6a2-4e11-8fa4-bff310c55346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:38.712181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34c65b-d6a2-4e11-8fa4-bff310c55346 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:38.713023 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a74c2-6227-4f7b-b18f-7e87133de381 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:38.724072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740ac57-51ca-44f9-b10a-4a3f94e01f44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:38.724757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740ac57-51ca-44f9-b10a-4a3f94e01f44 service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:38.725059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740ac57-51ca-44f9-b10a-4a3f94e01f44 service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:38.725350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740ac57-51ca-44f9-b10a-4a3f94e01f44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:38.735851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9740ac57-51ca-44f9-b10a-4a3f94e01f44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:38.758370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740ac57-51ca-44f9-b10a-4a3f94e01f44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:38.759992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740ac57-51ca-44f9-b10a-4a3f94e01f44 service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:38.761721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 860/1696] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:38 2022] POST /identity/v3/users => generated 279 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:39:38.763244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4deb5-b337-4a1d-b729-c6b63b3bc4a9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:38.763542 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4deb5-b337-4a1d-b729-c6b63b3bc4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:38.763816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4deb5-b337-4a1d-b729-c6b63b3bc4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:38.785463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 837/1697] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:39:38.786142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740ac57-51ca-44f9-b10a-4a3f94e01f44 service placement] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:38.806363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740ac57-51ca-44f9-b10a-4a3f94e01f44 service placement] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:38.811113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-11a4deb5-b337-4a1d-b729-c6b63b3bc4a9 None None] MFA Rules not processed for user `e56531920fc548a3a37f63ca0d1e5d82`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:38.812391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-11a4deb5-b337-4a1d-b729-c6b63b3bc4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:38.813125 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc80ee3-1ae0-4491-b188-27b287a4fe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:38.813815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc80ee3-1ae0-4491-b188-27b287a4fe66 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:38.814143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc80ee3-1ae0-4491-b188-27b287a4fe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:38.814421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc80ee3-1ae0-4491-b188-27b287a4fe66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:38.815167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc80ee3-1ae0-4491-b188-27b287a4fe66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:38.816811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc80ee3-1ae0-4491-b188-27b287a4fe66 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:38.848230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 838/1698] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:38.849753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc80ee3-1ae0-4491-b188-27b287a4fe66 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:38.867717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc80ee3-1ae0-4491-b188-27b287a4fe66 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:38.881750 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e112d4-dce3-4e8e-bb10-8f7e2685a980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:38.882652 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e112d4-dce3-4e8e-bb10-8f7e2685a980 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:38.883076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e112d4-dce3-4e8e-bb10-8f7e2685a980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:38.883401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e112d4-dce3-4e8e-bb10-8f7e2685a980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:38.885035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e112d4-dce3-4e8e-bb10-8f7e2685a980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:38.894156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e112d4-dce3-4e8e-bb10-8f7e2685a980 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:38.944351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 839/1699] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:38.947466 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e112d4-dce3-4e8e-bb10-8f7e2685a980 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:38.977955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e112d4-dce3-4e8e-bb10-8f7e2685a980 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:38.980332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 861/1700] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:39.009261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e69cf9-3439-4bee-996d-cc153d8e8f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:39.010207 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e69cf9-3439-4bee-996d-cc153d8e8f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:39.010492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e69cf9-3439-4bee-996d-cc153d8e8f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:39.010772 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e69cf9-3439-4bee-996d-cc153d8e8f45 admin admin] PATH_INFO: `/v3/projects/83d5ea686e98454bbe714611c77c69e6/users/4de9b84f323b4cddb99640a7f9625380/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:39.068617 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e69cf9-3439-4bee-996d-cc153d8e8f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83d5ea686e98454bbe714611c77c69e6, user_id=4de9b84f323b4cddb99640a7f9625380, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:39.077615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e69cf9-3439-4bee-996d-cc153d8e8f45 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:39.110406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 840/1701] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:38 2022] PUT /identity/v3/projects/83d5ea686e98454bbe714611c77c69e6/users/4de9b84f323b4cddb99640a7f9625380/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:39.125229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a61b6e-b376-4def-a5d3-1ed261306af3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:39.125522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a61b6e-b376-4def-a5d3-1ed261306af3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:39.125801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a61b6e-b376-4def-a5d3-1ed261306af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:39.186240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d2a61b6e-b376-4def-a5d3-1ed261306af3 None None] MFA Rules not processed for user `4de9b84f323b4cddb99640a7f9625380`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:39.187537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d2a61b6e-b376-4def-a5d3-1ed261306af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:39.246382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:39.330453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 862/1702] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:39.335087 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e69cf9-3439-4bee-996d-cc153d8e8f45 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:39.420088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e69cf9-3439-4bee-996d-cc153d8e8f45 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:39.435074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2e885-279c-4de9-9061-62907412ecfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:39.435778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2e885-279c-4de9-9061-62907412ecfd admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:39.436078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2e885-279c-4de9-9061-62907412ecfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:39.436882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2e885-279c-4de9-9061-62907412ecfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:39.437071 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2e885-279c-4de9-9061-62907412ecfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:39.443600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2e885-279c-4de9-9061-62907412ecfd admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:39.504970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 841/1703] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:39 2022] POST /identity/v3/projects => generated 357 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:39.507000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a61b6e-b376-4def-a5d3-1ed261306af3 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:39.524997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a61b6e-b376-4def-a5d3-1ed261306af3 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:39.646214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b1f0b-6303-4af1-ad4f-ca827b56265a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:39.647250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b1f0b-6303-4af1-ad4f-ca827b56265a admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:39.647721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b1f0b-6303-4af1-ad4f-ca827b56265a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:39.656299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b1f0b-6303-4af1-ad4f-ca827b56265a admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:39.657428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b1f0b-6303-4af1-ad4f-ca827b56265a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:39.660246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b1f0b-6303-4af1-ad4f-ca827b56265a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:39.727307 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 863/1704] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:39 2022] POST /identity/v3/users => generated 285 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:39.728919 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2e885-279c-4de9-9061-62907412ecfd admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:39.744921 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2e885-279c-4de9-9061-62907412ecfd admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:39.781971 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d317b5-0d8a-4601-9864-7da4ddeb4823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:39.782909 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d317b5-0d8a-4601-9864-7da4ddeb4823 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:39.783552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d317b5-0d8a-4601-9864-7da4ddeb4823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:39.783833 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d317b5-0d8a-4601-9864-7da4ddeb4823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:39.785566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d317b5-0d8a-4601-9864-7da4ddeb4823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:39.789036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d317b5-0d8a-4601-9864-7da4ddeb4823 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:39.800918 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bc79a-3e1c-4210-adf6-25409a0ced62 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:39.801533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bc79a-3e1c-4210-adf6-25409a0ced62 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:39.802084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bc79a-3e1c-4210-adf6-25409a0ced62 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:39.803948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 864/1705] 10.222.0.60 () {54 vars in 912 bytes} [Mon Jul 25 12:39:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:39:39.809781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 842/1706] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:39:39.811388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bc79a-3e1c-4210-adf6-25409a0ced62 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:39.826059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928ead9-ae4f-42b7-9db1-fa8d492cf2c4 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:39.826460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928ead9-ae4f-42b7-9db1-fa8d492cf2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:39.827214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928ead9-ae4f-42b7-9db1-fa8d492cf2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:39.841940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bc79a-3e1c-4210-adf6-25409a0ced62 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:39.854047 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8457ade-af41-41ac-8631-5676032e8f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:39.855084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8457ade-af41-41ac-8631-5676032e8f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:39.855466 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8457ade-af41-41ac-8631-5676032e8f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:39.855842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8457ade-af41-41ac-8631-5676032e8f87 admin admin] PATH_INFO: `/v3/projects/9d9913d9fc854aca86e525983408ab7e/users/9764d3a3cc6e4f22a3a2954a31070311/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:39.870865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f928ead9-ae4f-42b7-9db1-fa8d492cf2c4 None None] MFA Rules not processed for user `375d321d6de2494794254e50151c02c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:39.874157 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f928ead9-ae4f-42b7-9db1-fa8d492cf2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:39.906402 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8457ade-af41-41ac-8631-5676032e8f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d9913d9fc854aca86e525983408ab7e, user_id=9764d3a3cc6e4f22a3a2954a31070311, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:39.909955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8457ade-af41-41ac-8631-5676032e8f87 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:39.947333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 865/1707] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:39 2022] PUT /identity/v3/projects/9d9913d9fc854aca86e525983408ab7e/users/9764d3a3cc6e4f22a3a2954a31070311/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:39.948273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8457ade-af41-41ac-8631-5676032e8f87 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:39.966196 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:40.005162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 843/1708] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:40.027922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928ead9-ae4f-42b7-9db1-fa8d492cf2c4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:40.029755 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f928ead9-ae4f-42b7-9db1-fa8d492cf2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:40.050762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928ead9-ae4f-42b7-9db1-fa8d492cf2c4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:40.127619 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8457ade-af41-41ac-8631-5676032e8f87 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:40.133760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b887f-8f0b-4166-8040-997fae087e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:40.134731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b887f-8f0b-4166-8040-997fae087e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:40.135143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b887f-8f0b-4166-8040-997fae087e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:40.135538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b887f-8f0b-4166-8040-997fae087e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:40.136435 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b887f-8f0b-4166-8040-997fae087e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:40.138375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b887f-8f0b-4166-8040-997fae087e29 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:40.154355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 866/1709] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:40.159118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b887f-8f0b-4166-8040-997fae087e29 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:40.170633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106f041-dfb6-4d43-835c-8c05ece7ffb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:40.176532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106f041-dfb6-4d43-835c-8c05ece7ffb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:40.176707 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b887f-8f0b-4166-8040-997fae087e29 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:40.177083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106f041-dfb6-4d43-835c-8c05ece7ffb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:40.178208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106f041-dfb6-4d43-835c-8c05ece7ffb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:40.190067 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b59d7-7949-4c80-9e96-9324837cf3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:40.190998 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b59d7-7949-4c80-9e96-9324837cf3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:40.191281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b59d7-7949-4c80-9e96-9324837cf3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:40.191560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b59d7-7949-4c80-9e96-9324837cf3a8 admin admin] PATH_INFO: `/v3/projects/9d9913d9fc854aca86e525983408ab7e/users/9764d3a3cc6e4f22a3a2954a31070311/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:40.193743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b59d7-7949-4c80-9e96-9324837cf3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d9913d9fc854aca86e525983408ab7e, user_id=9764d3a3cc6e4f22a3a2954a31070311, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:40.196757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b59d7-7949-4c80-9e96-9324837cf3a8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:40.198857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106f041-dfb6-4d43-835c-8c05ece7ffb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:40.205729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106f041-dfb6-4d43-835c-8c05ece7ffb0 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:40.210012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 867/1710] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:40 2022] PUT /identity/v3/projects/9d9913d9fc854aca86e525983408ab7e/users/9764d3a3cc6e4f22a3a2954a31070311/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:40.220450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 844/1711] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:40.222591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d1139-7979-40ea-8ebb-ea11fe9db1a3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:40.222950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d1139-7979-40ea-8ebb-ea11fe9db1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:40.228943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d1139-7979-40ea-8ebb-ea11fe9db1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:40.274355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:40.279867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-933d1139-7979-40ea-8ebb-ea11fe9db1a3 None None] MFA Rules not processed for user `9764d3a3cc6e4f22a3a2954a31070311`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:40.282626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-933d1139-7979-40ea-8ebb-ea11fe9db1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:40.424222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 868/1712] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:40 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:39:40.438505 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106f041-dfb6-4d43-835c-8c05ece7ffb0 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:40.439987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2106f041-dfb6-4d43-835c-8c05ece7ffb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:40.572683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106f041-dfb6-4d43-835c-8c05ece7ffb0 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:40.692578 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee54cdb-d0b5-436a-a678-e46a255d2855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:40.693248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee54cdb-d0b5-436a-a678-e46a255d2855 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:40.693532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee54cdb-d0b5-436a-a678-e46a255d2855 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:40.693874 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee54cdb-d0b5-436a-a678-e46a255d2855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:40.720732 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee54cdb-d0b5-436a-a678-e46a255d2855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:40.722730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee54cdb-d0b5-436a-a678-e46a255d2855 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:40.745253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 845/1713] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:40 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:40.880894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d1139-7979-40ea-8ebb-ea11fe9db1a3 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:40.891593 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d1139-7979-40ea-8ebb-ea11fe9db1a3 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:40.900884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0bdd2-ce38-4e21-b41d-97dcf4c43bce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:40.901775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0bdd2-ce38-4e21-b41d-97dcf4c43bce service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:40.902253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0bdd2-ce38-4e21-b41d-97dcf4c43bce service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:40.902704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0bdd2-ce38-4e21-b41d-97dcf4c43bce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:40.977632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0bdd2-ce38-4e21-b41d-97dcf4c43bce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:40.979344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0bdd2-ce38-4e21-b41d-97dcf4c43bce service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:41.004145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 869/1714] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:40 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:42.731940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdf37c-8c76-4c51-ae6f-722920a1c5ef None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:42.733569 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdf37c-8c76-4c51-ae6f-722920a1c5ef None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:42.734067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdf37c-8c76-4c51-ae6f-722920a1c5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:42.979842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-23fdf37c-8c76-4c51-ae6f-722920a1c5ef None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:42.981458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-23fdf37c-8c76-4c51-ae6f-722920a1c5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:43.021832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7827-c07b-4879-8eec-5bf252eea257 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:43.022366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7827-c07b-4879-8eec-5bf252eea257 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:43.022901 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7827-c07b-4879-8eec-5bf252eea257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:43.050230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:43.070504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b3ff7827-c07b-4879-8eec-5bf252eea257 None None] MFA Rules not processed for user `1bc5799ac34349518854829ec2c259f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:43.078171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b3ff7827-c07b-4879-8eec-5bf252eea257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:43.220481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 846/1715] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:39:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:43.347680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 870/1716] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:43.491485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fdf37c-8c76-4c51-ae6f-722920a1c5ef None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:43.493689 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-23fdf37c-8c76-4c51-ae6f-722920a1c5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:43.722273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fdf37c-8c76-4c51-ae6f-722920a1c5ef None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:43.765996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24751e-f9a7-4c57-9fef-2e1595a0ec25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:43.767154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24751e-f9a7-4c57-9fef-2e1595a0ec25 service swift] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:43.768717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24751e-f9a7-4c57-9fef-2e1595a0ec25 service swift] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:43.769076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24751e-f9a7-4c57-9fef-2e1595a0ec25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:43.797358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24751e-f9a7-4c57-9fef-2e1595a0ec25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:43.800940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24751e-f9a7-4c57-9fef-2e1595a0ec25 service swift] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:43.823115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 847/1717] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:39:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:39:46.624011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a3094-527b-4986-9429-d0b67a5e3ae4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:46.624611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a3094-527b-4986-9429-d0b67a5e3ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:46.624611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a3094-527b-4986-9429-d0b67a5e3ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:46.733250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-727a3094-527b-4986-9429-d0b67a5e3ae4 None None] MFA Rules not processed for user `b0f3576844744c538a4755221c4added`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:46.738951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-727a3094-527b-4986-9429-d0b67a5e3ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:46.900752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:47.067046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 871/1718] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:47.083277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98d8f2-0f42-442f-8dc8-2ab07c0c1350 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:47.083848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98d8f2-0f42-442f-8dc8-2ab07c0c1350 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:47.083939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98d8f2-0f42-442f-8dc8-2ab07c0c1350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:47.165846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-fb98d8f2-0f42-442f-8dc8-2ab07c0c1350 None None] MFA Rules not processed for user `b0f3576844744c538a4755221c4added`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:47.167148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fb98d8f2-0f42-442f-8dc8-2ab07c0c1350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:47.200824 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 848/1719] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:47.465994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a3094-527b-4986-9429-d0b67a5e3ae4 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:47.468223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-727a3094-527b-4986-9429-d0b67a5e3ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:47.667137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc7124-bb58-4203-bc47-6984cb61c2b7 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:47.667415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc7124-bb58-4203-bc47-6984cb61c2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:47.667693 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc7124-bb58-4203-bc47-6984cb61c2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:47.734056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-0ffc7124-bb58-4203-bc47-6984cb61c2b7 None None] MFA Rules not processed for user `375d321d6de2494794254e50151c02c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:47.735621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0ffc7124-bb58-4203-bc47-6984cb61c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:47.889799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a3094-527b-4986-9429-d0b67a5e3ae4 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:47.909101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f384c-8895-4e2c-91b1-a0b178b702d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:47.909735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f384c-8895-4e2c-91b1-a0b178b702d5 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:47.910063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f384c-8895-4e2c-91b1-a0b178b702d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:47.910350 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f384c-8895-4e2c-91b1-a0b178b702d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:47.957297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f384c-8895-4e2c-91b1-a0b178b702d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:47.959098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f384c-8895-4e2c-91b1-a0b178b702d5 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:48.031418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 849/1721] 10.222.0.61 () {58 vars in 1015 bytes} [Mon Jul 25 12:39:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:39:48.031660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 872/1721] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:48.051621 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f384c-8895-4e2c-91b1-a0b178b702d5 service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:48.065091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f384c-8895-4e2c-91b1-a0b178b702d5 service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:48.086221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9219a1-0456-4286-a48b-3e7eb5c7ddea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:48.088217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9219a1-0456-4286-a48b-3e7eb5c7ddea service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:48.088479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9219a1-0456-4286-a48b-3e7eb5c7ddea service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:48.091618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9219a1-0456-4286-a48b-3e7eb5c7ddea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:48.104188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9219a1-0456-4286-a48b-3e7eb5c7ddea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:48.108825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9219a1-0456-4286-a48b-3e7eb5c7ddea service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:48.120907 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 873/1722] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:49.001467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc7124-bb58-4203-bc47-6984cb61c2b7 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:49.003615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0ffc7124-bb58-4203-bc47-6984cb61c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:49.138861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:49.222844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc7124-bb58-4203-bc47-6984cb61c2b7 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:49.236156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1d33b-242b-40c5-be17-f981a955cb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:49.237969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1d33b-242b-40c5-be17-f981a955cb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:49.240150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1d33b-242b-40c5-be17-f981a955cb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:49.242300 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1d33b-242b-40c5-be17-f981a955cb3c admin admin] PATH_INFO: `/v3/users/b0008e054713493aa956f632dfba0d4f` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:49.246059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1d33b-242b-40c5-be17-f981a955cb3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0008e054713493aa956f632dfba0d4f)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:49.252344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1d33b-242b-40c5-be17-f981a955cb3c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:49.382110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1ef1d33b-242b-40c5-be17-f981a955cb3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0008e054713493aa956f632dfba0d4f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:49.383016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1ef1d33b-242b-40c5-be17-f981a955cb3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0008e054713493aa956f632dfba0d4f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:49.384975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1ef1d33b-242b-40c5-be17-f981a955cb3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0008e054713493aa956f632dfba0d4f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:49.404243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1ef1d33b-242b-40c5-be17-f981a955cb3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0008e054713493aa956f632dfba0d4f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:49.424448 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1ef1d33b-242b-40c5-be17-f981a955cb3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0008e054713493aa956f632dfba0d4f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:49.483826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 850/1723] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:39:49 2022] DELETE /identity/v3/users/b0008e054713493aa956f632dfba0d4f => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:49.484634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbc055-a1c0-4024-af49-454795003d9e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:49.484634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbc055-a1c0-4024-af49-454795003d9e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:49.484634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbc055-a1c0-4024-af49-454795003d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:49.564722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-65cbc055-a1c0-4024-af49-454795003d9e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:49.573422 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-65cbc055-a1c0-4024-af49-454795003d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:49.775564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 874/1724] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:49.821865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bf14f-8547-4b6b-ae31-421b9fa2419f None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:49.822340 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bf14f-8547-4b6b-ae31-421b9fa2419f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:49.822722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bf14f-8547-4b6b-ae31-421b9fa2419f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:49.865993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbc055-a1c0-4024-af49-454795003d9e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:49.867953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-65cbc055-a1c0-4024-af49-454795003d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:49.890592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b95bf14f-8547-4b6b-ae31-421b9fa2419f None None] MFA Rules not processed for user `6f611c60491941ca8a0200212ab0c7a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:49.894934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b95bf14f-8547-4b6b-ae31-421b9fa2419f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:49.989971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:50.018314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbc055-a1c0-4024-af49-454795003d9e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:50.027495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01658cf8-8d83-4e3c-b22e-57d1f681a50c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:50.031705 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01658cf8-8d83-4e3c-b22e-57d1f681a50c service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:50.032150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01658cf8-8d83-4e3c-b22e-57d1f681a50c service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:50.032540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01658cf8-8d83-4e3c-b22e-57d1f681a50c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:50.071712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01658cf8-8d83-4e3c-b22e-57d1f681a50c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:50.073365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01658cf8-8d83-4e3c-b22e-57d1f681a50c service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:50.130544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 875/1725] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:50.227205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 851/1726] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:50.309473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01658cf8-8d83-4e3c-b22e-57d1f681a50c service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:50.311470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-01658cf8-8d83-4e3c-b22e-57d1f681a50c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:50.344175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01658cf8-8d83-4e3c-b22e-57d1f681a50c service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:50.362734 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767652f-7c00-447a-b66e-b99dd6b41e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:50.367558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767652f-7c00-447a-b66e-b99dd6b41e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:50.367558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767652f-7c00-447a-b66e-b99dd6b41e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:50.367558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767652f-7c00-447a-b66e-b99dd6b41e1a admin admin] PATH_INFO: `/v3/users/f089b2feef6a4b8599dd077ea809a25a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:50.370731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767652f-7c00-447a-b66e-b99dd6b41e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f089b2feef6a4b8599dd077ea809a25a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:50.375045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767652f-7c00-447a-b66e-b99dd6b41e1a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:50.523507 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c767652f-7c00-447a-b66e-b99dd6b41e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f089b2feef6a4b8599dd077ea809a25a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:50.543000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c767652f-7c00-447a-b66e-b99dd6b41e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f089b2feef6a4b8599dd077ea809a25a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:50.543910 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c767652f-7c00-447a-b66e-b99dd6b41e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f089b2feef6a4b8599dd077ea809a25a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:50.573413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c767652f-7c00-447a-b66e-b99dd6b41e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f089b2feef6a4b8599dd077ea809a25a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:50.576717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c767652f-7c00-447a-b66e-b99dd6b41e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f089b2feef6a4b8599dd077ea809a25a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:50.614777 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bf14f-8547-4b6b-ae31-421b9fa2419f None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:50.616437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b95bf14f-8547-4b6b-ae31-421b9fa2419f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:50.637210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 876/1727] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:39:50 2022] DELETE /identity/v3/users/f089b2feef6a4b8599dd077ea809a25a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:50.642228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6f9b7-7e73-401e-8010-681ba1d31e99 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:50.642840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6f9b7-7e73-401e-8010-681ba1d31e99 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:50.643341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6f9b7-7e73-401e-8010-681ba1d31e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:50.687387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d9c6f9b7-7e73-401e-8010-681ba1d31e99 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:50.689700 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d9c6f9b7-7e73-401e-8010-681ba1d31e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:50.784673 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:50.896204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 877/1728] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:50.923870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bf14f-8547-4b6b-ae31-421b9fa2419f None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:51.056799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6f9b7-7e73-401e-8010-681ba1d31e99 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:51.062196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d9c6f9b7-7e73-401e-8010-681ba1d31e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:51.183471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6bc5a-3715-4cbc-b6f4-335683d029fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:51.186966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6bc5a-3715-4cbc-b6f4-335683d029fb service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:51.189513 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6bc5a-3715-4cbc-b6f4-335683d029fb service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:51.190049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6bc5a-3715-4cbc-b6f4-335683d029fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:51.196570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-68d6bc5a-3715-4cbc-b6f4-335683d029fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:51.328304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6f9b7-7e73-401e-8010-681ba1d31e99 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:51.353049 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e912ac5-bb17-4d72-b023-ab1b03287554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:51.357040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e912ac5-bb17-4d72-b023-ab1b03287554 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:51.357335 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e912ac5-bb17-4d72-b023-ab1b03287554 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:51.358152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e912ac5-bb17-4d72-b023-ab1b03287554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:51.404891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e912ac5-bb17-4d72-b023-ab1b03287554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:51.409861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e912ac5-bb17-4d72-b023-ab1b03287554 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:51.425116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6bc5a-3715-4cbc-b6f4-335683d029fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:51.427493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6bc5a-3715-4cbc-b6f4-335683d029fb service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:51.454031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 878/1729] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:51.461431 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 852/1730] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 847 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:39:52.115973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e912ac5-bb17-4d72-b023-ab1b03287554 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:52.119472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8e912ac5-bb17-4d72-b023-ab1b03287554 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:52.133818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:52.157502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e912ac5-bb17-4d72-b023-ab1b03287554 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:52.164923 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:52.166737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:52.167192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:52.167682 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] PATH_INFO: `/v3/users/cdfc9e7c0c7441e5b643ffa8083d8c37` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:52.169338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdfc9e7c0c7441e5b643ffa8083d8c37)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:52.173471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:52.262374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdfc9e7c0c7441e5b643ffa8083d8c37'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:52.287157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdfc9e7c0c7441e5b643ffa8083d8c37'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:52.289274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdfc9e7c0c7441e5b643ffa8083d8c37'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:52.314965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdfc9e7c0c7441e5b643ffa8083d8c37'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:52.318045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdfc9e7c0c7441e5b643ffa8083d8c37'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:52.500147 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 879/1731] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:39:52 2022] DELETE /identity/v3/users/cdfc9e7c0c7441e5b643ffa8083d8c37 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:52.504225 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20da5d-e9d6-4630-9697-5171d1d70b23 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:52.505339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20da5d-e9d6-4630-9697-5171d1d70b23 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:52.505787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20da5d-e9d6-4630-9697-5171d1d70b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:52.564773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-fc20da5d-e9d6-4630-9697-5171d1d70b23 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:52.569199 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fc20da5d-e9d6-4630-9697-5171d1d70b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:52.784129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 853/1732] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:52.861123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:52.862762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:53.112945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0086729-499a-4c8c-816f-1bf4fe11bc97 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:53.124647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3258b5d-5cff-4c25-8a3b-4008c3ddfdf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:53.128864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3258b5d-5cff-4c25-8a3b-4008c3ddfdf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:53.130254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3258b5d-5cff-4c25-8a3b-4008c3ddfdf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:53.131058 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3258b5d-5cff-4c25-8a3b-4008c3ddfdf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:53.160929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3258b5d-5cff-4c25-8a3b-4008c3ddfdf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:53.165008 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3258b5d-5cff-4c25-8a3b-4008c3ddfdf3 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:53.227179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 880/1733] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:53.741791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca8b00-28c0-4497-91d2-434178400718 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:53.742741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca8b00-28c0-4497-91d2-434178400718 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:53.743188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca8b00-28c0-4497-91d2-434178400718 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:53.745022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 854/1734] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:39:54.095346 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d75a3-06e9-4fc6-be0e-bb0e95347af2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:54.095987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d75a3-06e9-4fc6-be0e-bb0e95347af2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:54.096558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d75a3-06e9-4fc6-be0e-bb0e95347af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:54.158727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c38d75a3-06e9-4fc6-be0e-bb0e95347af2 None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:54.162977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c38d75a3-06e9-4fc6-be0e-bb0e95347af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:54.375558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca8b00-28c0-4497-91d2-434178400718 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:54.378702 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a7ca8b00-28c0-4497-91d2-434178400718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:54.459931 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca8b00-28c0-4497-91d2-434178400718 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:54.483859 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:54.492453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:54.493719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:54.495133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] PATH_INFO: `/v3/projects/8e785d3c68c94bf48a10c8e556d6b993` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:54.504549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e785d3c68c94bf48a10c8e556d6b993)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:54.508293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 881/1735] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:39:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:54.509545 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d75a3-06e9-4fc6-be0e-bb0e95347af2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:54.512950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c38d75a3-06e9-4fc6-be0e-bb0e95347af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:54.517548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:54.573545 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d75a3-06e9-4fc6-be0e-bb0e95347af2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:54.598814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:54.601069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:54.604268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:54.605118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] PATH_INFO: `/v3/projects/cb082229375f42198cba0b84485bd7f5` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:54.607912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb082229375f42198cba0b84485bd7f5)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:54.611935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:54.617133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:54.627973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:54.767916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e785d3c68c94bf48a10c8e556d6b993', 'request_id': 'req-8e730a43-0583-4916-b0ee-8e46ceb54a68'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:54.819827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e785d3c68c94bf48a10c8e556d6b993', 'request_id': 'req-8e730a43-0583-4916-b0ee-8e46ceb54a68'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:54.843290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb082229375f42198cba0b84485bd7f5', 'request_id': 'req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:54.869019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb082229375f42198cba0b84485bd7f5', 'request_id': 'req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:54.872459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e785d3c68c94bf48a10c8e556d6b993', 'request_id': 'req-8e730a43-0583-4916-b0ee-8e46ceb54a68'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:54.893603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb082229375f42198cba0b84485bd7f5', 'request_id': 'req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:54.946242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] The token cache is being invalidate because project cb082229375f42198cba0b84485bd7f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:39:54.950323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:54.951379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:54.952795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] The token cache is being invalidate because project 8e785d3c68c94bf48a10c8e556d6b993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:39:54.954439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:54.957172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:54.961490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 882/1736] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:39:54 2022] DELETE /identity/v3/projects/cb082229375f42198cba0b84485bd7f5 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:54.962863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:54.966574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:54.968237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 855/1737] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:39:54 2022] DELETE /identity/v3/projects/8e785d3c68c94bf48a10c8e556d6b993 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:54.972394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:54.976927 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:55.103943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e730a43-0583-4916-b0ee-8e46ceb54a68 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:55.250616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bad6c6-cecf-4e0c-9202-25c6556c5116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:55.251770 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bad6c6-cecf-4e0c-9202-25c6556c5116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:55.252167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bad6c6-cecf-4e0c-9202-25c6556c5116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:55.252580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bad6c6-cecf-4e0c-9202-25c6556c5116 admin admin] PATH_INFO: `/v3/users/a357371ffbf44038b660bc924b2fef27` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:55.255397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bad6c6-cecf-4e0c-9202-25c6556c5116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a357371ffbf44038b660bc924b2fef27)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:55.260721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bad6c6-cecf-4e0c-9202-25c6556c5116 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:55.278749 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693aa496-f7d6-4955-a7f7-3b5ee6499b3c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:55.321379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:55.322758 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:55.324279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:55.324825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] PATH_INFO: `/v3/users/2123c46078684ad8b3a4045114bf817b` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:55.328652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2123c46078684ad8b3a4045114bf817b)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:55.331786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:55.386435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-63bad6c6-cecf-4e0c-9202-25c6556c5116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a357371ffbf44038b660bc924b2fef27'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:55.388200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-63bad6c6-cecf-4e0c-9202-25c6556c5116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a357371ffbf44038b660bc924b2fef27'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:55.392414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-63bad6c6-cecf-4e0c-9202-25c6556c5116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a357371ffbf44038b660bc924b2fef27'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:55.428142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:55.442636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-63bad6c6-cecf-4e0c-9202-25c6556c5116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a357371ffbf44038b660bc924b2fef27'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:55.458226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-63bad6c6-cecf-4e0c-9202-25c6556c5116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a357371ffbf44038b660bc924b2fef27'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:55.496447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 856/1738] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:39:54 2022] DELETE /identity/v3/users/a357371ffbf44038b660bc924b2fef27 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:55.513811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2123c46078684ad8b3a4045114bf817b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:55.531367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2123c46078684ad8b3a4045114bf817b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:55.532220 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2123c46078684ad8b3a4045114bf817b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:55.551537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2123c46078684ad8b3a4045114bf817b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:55.552344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2123c46078684ad8b3a4045114bf817b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:55.590767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d4859-e314-4a23-894a-e8a40a52711c None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:55.591174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d4859-e314-4a23-894a-e8a40a52711c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:55.591645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d4859-e314-4a23-894a-e8a40a52711c None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:55.593674 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 857/1739] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:39:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:39:55.645943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 883/1740] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:39:54 2022] DELETE /identity/v3/users/2123c46078684ad8b3a4045114bf817b => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:55.898984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d326f-4a9f-4506-91ee-da334535cab2 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:55.899575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d326f-4a9f-4506-91ee-da334535cab2 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:55.899852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d326f-4a9f-4506-91ee-da334535cab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:55.949807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-e77d326f-4a9f-4506-91ee-da334535cab2 None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:55.952394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e77d326f-4a9f-4506-91ee-da334535cab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:55.961889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:55.963353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:55.999297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:56.143705 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc8f07-ad62-491c-b5a4-8e66fadd753f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:56.176281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:56.179558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:56.183928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:56.184416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] PATH_INFO: `/v3/projects/2bcc5a12b5ef476d944b824008f03014` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:56.189840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bcc5a12b5ef476d944b824008f03014)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:56.192330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 858/1741] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:39:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:56.194901 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:56.287512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d326f-4a9f-4506-91ee-da334535cab2 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:56.296387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e77d326f-4a9f-4506-91ee-da334535cab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:56.314442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d326f-4a9f-4506-91ee-da334535cab2 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:56.322405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4f098-cc12-492d-90c2-65aaeb437a7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:56.323232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4f098-cc12-492d-90c2-65aaeb437a7b service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:56.323694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4f098-cc12-492d-90c2-65aaeb437a7b service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:56.324084 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4f098-cc12-492d-90c2-65aaeb437a7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:56.329447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7df4f098-cc12-492d-90c2-65aaeb437a7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:56.358251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4f098-cc12-492d-90c2-65aaeb437a7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:56.360382 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4f098-cc12-492d-90c2-65aaeb437a7b service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:56.370226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bcc5a12b5ef476d944b824008f03014', 'request_id': 'req-4fe8ea8c-d4c4-4976-8c24-024977294808'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:56.392883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bcc5a12b5ef476d944b824008f03014', 'request_id': 'req-4fe8ea8c-d4c4-4976-8c24-024977294808'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:56.413903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 859/1742] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:39:56.414158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bcc5a12b5ef476d944b824008f03014', 'request_id': 'req-4fe8ea8c-d4c4-4976-8c24-024977294808'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:56.449382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] The token cache is being invalidate because project 2bcc5a12b5ef476d944b824008f03014 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:39:56.451109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:56.453804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:56.461556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 884/1743] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:39:55 2022] DELETE /identity/v3/projects/2bcc5a12b5ef476d944b824008f03014 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:39:56.468074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715c088-7867-4c97-8f5c-96f030864728 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:56.468427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715c088-7867-4c97-8f5c-96f030864728 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:56.468844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715c088-7867-4c97-8f5c-96f030864728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:56.543302 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f715c088-7867-4c97-8f5c-96f030864728 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:56.550083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f715c088-7867-4c97-8f5c-96f030864728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:56.744845 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:56.747149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:56.901609 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 860/1744] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:56.906067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcd6a4-fb9b-4d94-beca-de9a847526b0 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:56.909587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcd6a4-fb9b-4d94-beca-de9a847526b0 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:56.915128 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcd6a4-fb9b-4d94-beca-de9a847526b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:56.986717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-1ebcd6a4-fb9b-4d94-beca-de9a847526b0 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:56.989117 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-1ebcd6a4-fb9b-4d94-beca-de9a847526b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:57.017984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 861/1745] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:57.019366 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcd6a4-fb9b-4d94-beca-de9a847526b0 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:57.039745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcd6a4-fb9b-4d94-beca-de9a847526b0 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:57.042866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ea8c-d4c4-4976-8c24-024977294808 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:57.051004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ac7ba-44fb-473d-af72-ba660e4772df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:57.052656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ac7ba-44fb-473d-af72-ba660e4772df admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:57.053585 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ac7ba-44fb-473d-af72-ba660e4772df admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:57.054413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ac7ba-44fb-473d-af72-ba660e4772df admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:57.056680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a2daf-5785-4311-aacd-8feb71276dbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:57.058273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a2daf-5785-4311-aacd-8feb71276dbe service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:57.058666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ac7ba-44fb-473d-af72-ba660e4772df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:57.058970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a2daf-5785-4311-aacd-8feb71276dbe service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:57.059070 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ac7ba-44fb-473d-af72-ba660e4772df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:57.059557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a2daf-5785-4311-aacd-8feb71276dbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:57.061721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ac7ba-44fb-473d-af72-ba660e4772df admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:57.062592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f59a2daf-5785-4311-aacd-8feb71276dbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:57.088906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a2daf-5785-4311-aacd-8feb71276dbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:57.091935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a2daf-5785-4311-aacd-8feb71276dbe service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:57.098132 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 862/1746] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:39:57.099319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ac7ba-44fb-473d-af72-ba660e4772df admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:57.111519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ac7ba-44fb-473d-af72-ba660e4772df admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:57.121097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3684ca-6d09-4d15-9f61-cdeccadc3052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:57.121841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3684ca-6d09-4d15-9f61-cdeccadc3052 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:57.122190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3684ca-6d09-4d15-9f61-cdeccadc3052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:57.122490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3684ca-6d09-4d15-9f61-cdeccadc3052 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:57.123967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3684ca-6d09-4d15-9f61-cdeccadc3052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:57.133784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 885/1747] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:39:57.154048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3684ca-6d09-4d15-9f61-cdeccadc3052 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:57.265313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a2daf-5785-4311-aacd-8feb71276dbe service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:57.271099 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 863/1748] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:57 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:57.281509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a2daf-5785-4311-aacd-8feb71276dbe service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:57.382207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:57.463847 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313eee5-2833-46c7-b058-0610de1a9537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:57.464756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313eee5-2833-46c7-b058-0610de1a9537 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:57.465179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313eee5-2833-46c7-b058-0610de1a9537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:57.465671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313eee5-2833-46c7-b058-0610de1a9537 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:57.467155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313eee5-2833-46c7-b058-0610de1a9537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:57.472272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313eee5-2833-46c7-b058-0610de1a9537 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:57.524126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 886/1749] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:57 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:39:57.528601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3684ca-6d09-4d15-9f61-cdeccadc3052 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:57.543726 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3684ca-6d09-4d15-9f61-cdeccadc3052 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:57.559514 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c84d40-d898-4789-a0a6-e83ec7b65aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:57.560362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c84d40-d898-4789-a0a6-e83ec7b65aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:57.560883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c84d40-d898-4789-a0a6-e83ec7b65aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:57.561294 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c84d40-d898-4789-a0a6-e83ec7b65aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:57.562175 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c84d40-d898-4789-a0a6-e83ec7b65aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:57.563906 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c84d40-d898-4789-a0a6-e83ec7b65aa3 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:57.571072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:57.590444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313eee5-2833-46c7-b058-0610de1a9537 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:57.600920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 864/1750] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:57.617101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313eee5-2833-46c7-b058-0610de1a9537 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:57.642358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b19d34-e434-4d95-a29a-2243eacbe147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:57.643065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b19d34-e434-4d95-a29a-2243eacbe147 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:57.643352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b19d34-e434-4d95-a29a-2243eacbe147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:57.648208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b19d34-e434-4d95-a29a-2243eacbe147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:57.649168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b19d34-e434-4d95-a29a-2243eacbe147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:57.650957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b19d34-e434-4d95-a29a-2243eacbe147 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:57.677799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 887/1751] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:39:57.682083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c84d40-d898-4789-a0a6-e83ec7b65aa3 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:57.700073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c84d40-d898-4789-a0a6-e83ec7b65aa3 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:57.715605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593434f-09c1-4b02-b000-7e3063985e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:57.716592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593434f-09c1-4b02-b000-7e3063985e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:57.717414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593434f-09c1-4b02-b000-7e3063985e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:57.717763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593434f-09c1-4b02-b000-7e3063985e17 admin admin] PATH_INFO: `/v3/projects/d12806b9ad9546fcb32a56e76ced516f/users/abbb41e2fc4b48f79866c0ffb1a792c1/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:57.777508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593434f-09c1-4b02-b000-7e3063985e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d12806b9ad9546fcb32a56e76ced516f, user_id=abbb41e2fc4b48f79866c0ffb1a792c1, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:57.781847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593434f-09c1-4b02-b000-7e3063985e17 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:57.833342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 865/1752] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:39:57 2022] PUT /identity/v3/projects/d12806b9ad9546fcb32a56e76ced516f/users/abbb41e2fc4b48f79866c0ffb1a792c1/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:57.848840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3a475-8164-48f6-901d-3a9b940060b0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:57.848840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3a475-8164-48f6-901d-3a9b940060b0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:57.848994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3a475-8164-48f6-901d-3a9b940060b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:57.925156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a8b3a475-8164-48f6-901d-3a9b940060b0 None None] MFA Rules not processed for user `abbb41e2fc4b48f79866c0ffb1a792c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:57.926418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a8b3a475-8164-48f6-901d-3a9b940060b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:58.148822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 888/1753] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:58.162177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593434f-09c1-4b02-b000-7e3063985e17 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:58.287285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3a475-8164-48f6-901d-3a9b940060b0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:58.289160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a8b3a475-8164-48f6-901d-3a9b940060b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:58.353592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593434f-09c1-4b02-b000-7e3063985e17 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:58.368405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529885c7-11fd-438e-9bc2-30f7b94470de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:58.369836 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529885c7-11fd-438e-9bc2-30f7b94470de service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:58.370368 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529885c7-11fd-438e-9bc2-30f7b94470de service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:58.370879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529885c7-11fd-438e-9bc2-30f7b94470de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:58.406370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529885c7-11fd-438e-9bc2-30f7b94470de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:58.409415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529885c7-11fd-438e-9bc2-30f7b94470de service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:58.439144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 866/1754] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:39:58.556354 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3a475-8164-48f6-901d-3a9b940060b0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:58.570555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:58.572333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:58.572778 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:58.573394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12 admin admin] PATH_INFO: `/v3/projects/47512038402f4e7fa5acd766bd8dc11c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:58.575194 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47512038402f4e7fa5acd766bd8dc11c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:58.582275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:58.680167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529885c7-11fd-438e-9bc2-30f7b94470de service nova] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:58.681800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-529885c7-11fd-438e-9bc2-30f7b94470de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:58.722845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529885c7-11fd-438e-9bc2-30f7b94470de service nova] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:58.747593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:58.747593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:58.747593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:58.750446 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] PATH_INFO: `/v3/projects/9856cf40febc4e79b9e5e0a00cf27be0` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:58.753014 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:58.758051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9856cf40febc4e79b9e5e0a00cf27be0)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:58.764204 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:58.782495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47512038402f4e7fa5acd766bd8dc11c', 'request_id': 'req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:58.800270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47512038402f4e7fa5acd766bd8dc11c', 'request_id': 'req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:58.813861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47512038402f4e7fa5acd766bd8dc11c', 'request_id': 'req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:58.841109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12 admin admin] The token cache is being invalidate because project 47512038402f4e7fa5acd766bd8dc11c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:39:58.841564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:58.842356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d94c2505-ae22-4ce6-83bc-e71a3eb12d12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:58.847732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 889/1755] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:39:58 2022] DELETE /identity/v3/projects/47512038402f4e7fa5acd766bd8dc11c => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:39:58.853888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba93281-ae83-484e-a652-3dffb224f426 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:58.854243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba93281-ae83-484e-a652-3dffb224f426 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:58.854534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba93281-ae83-484e-a652-3dffb224f426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:58.880973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9856cf40febc4e79b9e5e0a00cf27be0', 'request_id': 'req-03f645f6-db46-4b7c-81b2-9dadc270489e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:58.888955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9856cf40febc4e79b9e5e0a00cf27be0', 'request_id': 'req-03f645f6-db46-4b7c-81b2-9dadc270489e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:58.895639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9856cf40febc4e79b9e5e0a00cf27be0', 'request_id': 'req-03f645f6-db46-4b7c-81b2-9dadc270489e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:58.918457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] The token cache is being invalidate because project 9856cf40febc4e79b9e5e0a00cf27be0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:39:58.919284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:58.921502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:58.929296 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 867/1756] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:39:58 2022] DELETE /identity/v3/projects/9856cf40febc4e79b9e5e0a00cf27be0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:58.930439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:58.934564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-bba93281-ae83-484e-a652-3dffb224f426 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:58.936262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:58.936380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bba93281-ae83-484e-a652-3dffb224f426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:59.055726 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:59.105050 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f645f6-db46-4b7c-81b2-9dadc270489e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:59.135559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:59.135559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:59.140341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:59.141970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] PATH_INFO: `/v3/users/abbb41e2fc4b48f79866c0ffb1a792c1` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:59.146474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abbb41e2fc4b48f79866c0ffb1a792c1)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:59.150929 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:59.162423 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 890/1757] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:59.163923 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d8ad6-14bf-49f0-8dad-dbd5c9eb2c93 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:59.164230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d8ad6-14bf-49f0-8dad-dbd5c9eb2c93 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:59.164526 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d8ad6-14bf-49f0-8dad-dbd5c9eb2c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:59.208747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d29d8ad6-14bf-49f0-8dad-dbd5c9eb2c93 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:59.210033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d29d8ad6-14bf-49f0-8dad-dbd5c9eb2c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:59.241571 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 891/1758] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:59.243149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f6de7-edb4-4998-83ef-a964499348d0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:59.243434 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f6de7-edb4-4998-83ef-a964499348d0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:59.243716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f6de7-edb4-4998-83ef-a964499348d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:59.326280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-733f6de7-edb4-4998-83ef-a964499348d0 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:59.329501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-733f6de7-edb4-4998-83ef-a964499348d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:59.338997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abbb41e2fc4b48f79866c0ffb1a792c1'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:59.340179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abbb41e2fc4b48f79866c0ffb1a792c1'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:59.341250 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abbb41e2fc4b48f79866c0ffb1a792c1'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:59.367909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 892/1759] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:59.369703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299c524-c11d-4ebb-9420-af0dd6a8c87c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:59.370233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299c524-c11d-4ebb-9420-af0dd6a8c87c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:59.370717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299c524-c11d-4ebb-9420-af0dd6a8c87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:59.374604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abbb41e2fc4b48f79866c0ffb1a792c1'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:59.388452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abbb41e2fc4b48f79866c0ffb1a792c1'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:39:59.438617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7299c524-c11d-4ebb-9420-af0dd6a8c87c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:59.441754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7299c524-c11d-4ebb-9420-af0dd6a8c87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:59.450028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 868/1760] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:39:58 2022] DELETE /identity/v3/users/abbb41e2fc4b48f79866c0ffb1a792c1 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:39:59.451015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:59.574999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 893/1761] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:59.577052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab014b-fd8f-4e59-a806-e91f34f8049e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:59.578048 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab014b-fd8f-4e59-a806-e91f34f8049e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:59.578452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab014b-fd8f-4e59-a806-e91f34f8049e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:59.614860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6dab014b-fd8f-4e59-a806-e91f34f8049e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:39:59.616520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6dab014b-fd8f-4e59-a806-e91f34f8049e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:59.638350 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 894/1762] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:39:59.639437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab014b-fd8f-4e59-a806-e91f34f8049e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:59.655156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab014b-fd8f-4e59-a806-e91f34f8049e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:59.665842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2e7da-eb6c-48e6-bb32-3dbf8c0a7d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:59.667217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2e7da-eb6c-48e6-bb32-3dbf8c0a7d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:59.667947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2e7da-eb6c-48e6-bb32-3dbf8c0a7d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:59.668611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2e7da-eb6c-48e6-bb32-3dbf8c0a7d2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:59.671546 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2e7da-eb6c-48e6-bb32-3dbf8c0a7d2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:59.672508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2e7da-eb6c-48e6-bb32-3dbf8c0a7d2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:59.676358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36c0c9-cfc9-41a1-bf9d-e8c8a1ad02f0 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:59.678801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2e7da-eb6c-48e6-bb32-3dbf8c0a7d2f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:59.712233 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b102de7-8fb2-49db-92b5-d900cb8e349c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:59.713209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102de7-8fb2-49db-92b5-d900cb8e349c admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:59.719919 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102de7-8fb2-49db-92b5-d900cb8e349c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:59.720415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102de7-8fb2-49db-92b5-d900cb8e349c admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:59.721560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b102de7-8fb2-49db-92b5-d900cb8e349c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:39:59.722112 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b102de7-8fb2-49db-92b5-d900cb8e349c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:59.724079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b102de7-8fb2-49db-92b5-d900cb8e349c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:59.728073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 895/1763] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:39:59.729858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2e7da-eb6c-48e6-bb32-3dbf8c0a7d2f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:59.767456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2e7da-eb6c-48e6-bb32-3dbf8c0a7d2f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:59.783861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 869/1764] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:39:59.785226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b102de7-8fb2-49db-92b5-d900cb8e349c admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:59.788034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b565b-6699-4c1f-a61a-e6d1da70d5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:59.788891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b565b-6699-4c1f-a61a-e6d1da70d5de admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:59.789679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b565b-6699-4c1f-a61a-e6d1da70d5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:59.789679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b565b-6699-4c1f-a61a-e6d1da70d5de admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:59.790152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b565b-6699-4c1f-a61a-e6d1da70d5de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:59.792075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b565b-6699-4c1f-a61a-e6d1da70d5de admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:59.812415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b102de7-8fb2-49db-92b5-d900cb8e349c admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:59.824791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164ba6b-7e32-45e2-847b-b5e40546864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:39:59.826732 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164ba6b-7e32-45e2-847b-b5e40546864f admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:39:59.827919 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164ba6b-7e32-45e2-847b-b5e40546864f admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:39:59.828266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164ba6b-7e32-45e2-847b-b5e40546864f admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:39:59.832328 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164ba6b-7e32-45e2-847b-b5e40546864f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:39:59.835843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164ba6b-7e32-45e2-847b-b5e40546864f admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:39:59.860578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 896/1765] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:59 2022] POST /identity/v3/projects => generated 399 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:59.861512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b565b-6699-4c1f-a61a-e6d1da70d5de admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:59.865872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a97b565b-6699-4c1f-a61a-e6d1da70d5de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:39:59.903813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:39:59.919568 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 870/1766] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:39:59 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:39:59.920827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164ba6b-7e32-45e2-847b-b5e40546864f admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:39:59.976191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b565b-6699-4c1f-a61a-e6d1da70d5de admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:39:59.978484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:00.034659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164ba6b-7e32-45e2-847b-b5e40546864f admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:00.042658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7673fc-e6f3-49c1-93fa-f8592ed9a3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.043795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7673fc-e6f3-49c1-93fa-f8592ed9a3ee admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.044083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7673fc-e6f3-49c1-93fa-f8592ed9a3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.044357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7673fc-e6f3-49c1-93fa-f8592ed9a3ee admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.045757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7673fc-e6f3-49c1-93fa-f8592ed9a3ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.053287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7673fc-e6f3-49c1-93fa-f8592ed9a3ee admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:00.059834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2de7-c0b4-4809-8fcf-bb1ee932a550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.061186 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2de7-c0b4-4809-8fcf-bb1ee932a550 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.061562 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2de7-c0b4-4809-8fcf-bb1ee932a550 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.061975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2de7-c0b4-4809-8fcf-bb1ee932a550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.085384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d2de7-c0b4-4809-8fcf-bb1ee932a550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.092910 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d2de7-c0b4-4809-8fcf-bb1ee932a550 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:00.115948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 871/1768] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:39:59 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:00.116802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7673fc-e6f3-49c1-93fa-f8592ed9a3ee admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:00.119327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 897/1768] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:39:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:00.127360 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2de7-c0b4-4809-8fcf-bb1ee932a550 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:00.137978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2de7-c0b4-4809-8fcf-bb1ee932a550 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:00.142402 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7673fc-e6f3-49c1-93fa-f8592ed9a3ee admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:00.146723 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877e688-9616-4519-9a48-9a2704a18d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.147780 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877e688-9616-4519-9a48-9a2704a18d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.148129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877e688-9616-4519-9a48-9a2704a18d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.153820 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877e688-9616-4519-9a48-9a2704a18d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.153820 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877e688-9616-4519-9a48-9a2704a18d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.153820 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877e688-9616-4519-9a48-9a2704a18d05 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:00.156799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4372b-7845-4309-abad-7ed34b581bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.157512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4372b-7845-4309-abad-7ed34b581bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.157880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4372b-7845-4309-abad-7ed34b581bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.158201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4372b-7845-4309-abad-7ed34b581bc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.158997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4372b-7845-4309-abad-7ed34b581bc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.160940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4372b-7845-4309-abad-7ed34b581bc2 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:00.176932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 898/1769] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:00.178347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877e688-9616-4519-9a48-9a2704a18d05 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:00.190925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877e688-9616-4519-9a48-9a2704a18d05 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:00.202803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010590d-752d-4442-bd19-8f5838c01472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.203608 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010590d-752d-4442-bd19-8f5838c01472 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.204051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010590d-752d-4442-bd19-8f5838c01472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.204725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010590d-752d-4442-bd19-8f5838c01472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.205429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010590d-752d-4442-bd19-8f5838c01472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.215393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010590d-752d-4442-bd19-8f5838c01472 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:00.225888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 872/1770] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:00 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:00.231223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4372b-7845-4309-abad-7ed34b581bc2 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:00.239973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 899/1771] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:00.244024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010590d-752d-4442-bd19-8f5838c01472 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:00.247320 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4372b-7845-4309-abad-7ed34b581bc2 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:00.255209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010590d-752d-4442-bd19-8f5838c01472 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:00.267524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7842fb9-ad52-411e-8e13-e79767b250fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.268447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7842fb9-ad52-411e-8e13-e79767b250fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.268727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7842fb9-ad52-411e-8e13-e79767b250fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.269053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7842fb9-ad52-411e-8e13-e79767b250fa admin admin] PATH_INFO: `/v3/projects/9e0c8d7c32674536907ba51031b5e05e/users/493b1466e40a41319e02bc6711b3bb75/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.280835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a077576-1ff5-48b8-8928-da636e78b5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.281729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a077576-1ff5-48b8-8928-da636e78b5ea admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.284303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a077576-1ff5-48b8-8928-da636e78b5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.284303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a077576-1ff5-48b8-8928-da636e78b5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.284303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a077576-1ff5-48b8-8928-da636e78b5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.285619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a077576-1ff5-48b8-8928-da636e78b5ea admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:00.293724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7842fb9-ad52-411e-8e13-e79767b250fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0c8d7c32674536907ba51031b5e05e, user_id=493b1466e40a41319e02bc6711b3bb75, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.298011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7842fb9-ad52-411e-8e13-e79767b250fa admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:00.317576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 873/1772] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:00.323399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a077576-1ff5-48b8-8928-da636e78b5ea admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:00.328026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 900/1773] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:00 2022] PUT /identity/v3/projects/9e0c8d7c32674536907ba51031b5e05e/users/493b1466e40a41319e02bc6711b3bb75/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:00.338655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3bc5d-55cf-4ada-bd2c-084c77d0bcbb None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.338948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3bc5d-55cf-4ada-bd2c-084c77d0bcbb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.339230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3bc5d-55cf-4ada-bd2c-084c77d0bcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.355239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a077576-1ff5-48b8-8928-da636e78b5ea admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:00.403192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-97d3bc5d-55cf-4ada-bd2c-084c77d0bcbb None None] MFA Rules not processed for user `493b1466e40a41319e02bc6711b3bb75`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:00.405362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-97d3bc5d-55cf-4ada-bd2c-084c77d0bcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:00.507600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25480e21-cacd-4f7d-840b-59c414d3b590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.508291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25480e21-cacd-4f7d-840b-59c414d3b590 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.508567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25480e21-cacd-4f7d-840b-59c414d3b590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.508838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25480e21-cacd-4f7d-840b-59c414d3b590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.512353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25480e21-cacd-4f7d-840b-59c414d3b590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.518690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25480e21-cacd-4f7d-840b-59c414d3b590 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:00.550749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 874/1774] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:00.551691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25480e21-cacd-4f7d-840b-59c414d3b590 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:00.580607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25480e21-cacd-4f7d-840b-59c414d3b590 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:00.587755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 901/1775] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:00.592290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3bc5d-55cf-4ada-bd2c-084c77d0bcbb None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:00.594432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d45a2f-4368-4547-b81e-b68b1f6fa44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.595445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d45a2f-4368-4547-b81e-b68b1f6fa44b admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.595914 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d45a2f-4368-4547-b81e-b68b1f6fa44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.596258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d45a2f-4368-4547-b81e-b68b1f6fa44b admin admin] PATH_INFO: `/v3/projects/3f20ce7126484948b2ebd2c0a4985ec6/users/8a34d144c07b4d47b78ef8249aeaf46a/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.615131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d45a2f-4368-4547-b81e-b68b1f6fa44b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f20ce7126484948b2ebd2c0a4985ec6, user_id=8a34d144c07b4d47b78ef8249aeaf46a, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.618946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d45a2f-4368-4547-b81e-b68b1f6fa44b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:00.619067 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3bc5d-55cf-4ada-bd2c-084c77d0bcbb None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:00.630323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5e7c2-b7d8-43a0-8b23-312136e76b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.631218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5e7c2-b7d8-43a0-8b23-312136e76b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.632117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5e7c2-b7d8-43a0-8b23-312136e76b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.633119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5e7c2-b7d8-43a0-8b23-312136e76b57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.635168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5e7c2-b7d8-43a0-8b23-312136e76b57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.640182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5e7c2-b7d8-43a0-8b23-312136e76b57 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:00.650926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 875/1776] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:00 2022] PUT /identity/v3/projects/3f20ce7126484948b2ebd2c0a4985ec6/users/8a34d144c07b4d47b78ef8249aeaf46a/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:00.662310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812fc4d-3f07-45c7-9ce2-dc9224694e80 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.662599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812fc4d-3f07-45c7-9ce2-dc9224694e80 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.663301 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812fc4d-3f07-45c7-9ce2-dc9224694e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.692626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:00.702739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-c812fc4d-3f07-45c7-9ce2-dc9224694e80 None None] MFA Rules not processed for user `8a34d144c07b4d47b78ef8249aeaf46a`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:00.703981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c812fc4d-3f07-45c7-9ce2-dc9224694e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:00.730597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 902/1777] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:00 2022] POST /identity/v3/projects => generated 401 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:00.736092 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5e7c2-b7d8-43a0-8b23-312136e76b57 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:00.839377 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:00.854656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 876/1778] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:00.855792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812fc4d-3f07-45c7-9ce2-dc9224694e80 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:00.858617 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c812fc4d-3f07-45c7-9ce2-dc9224694e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:00.869978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5e7c2-b7d8-43a0-8b23-312136e76b57 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:00.880372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f91f0-4e74-4878-86b7-09fef6f07c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.881070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f91f0-4e74-4878-86b7-09fef6f07c2b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.881421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f91f0-4e74-4878-86b7-09fef6f07c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.881614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f91f0-4e74-4878-86b7-09fef6f07c2b admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.883525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f91f0-4e74-4878-86b7-09fef6f07c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.885785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f91f0-4e74-4878-86b7-09fef6f07c2b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:00.940208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 903/1779] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:00 2022] POST /identity/v3/users => generated 307 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:00.941281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f91f0-4e74-4878-86b7-09fef6f07c2b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:00.961340 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f91f0-4e74-4878-86b7-09fef6f07c2b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:00.966113 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812fc4d-3f07-45c7-9ce2-dc9224694e80 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:00.968001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa72af5-ef65-470f-a4a9-1ff701351e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.968722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa72af5-ef65-470f-a4a9-1ff701351e36 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.969005 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa72af5-ef65-470f-a4a9-1ff701351e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.969327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa72af5-ef65-470f-a4a9-1ff701351e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.970103 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa72af5-ef65-470f-a4a9-1ff701351e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.971586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:00.972055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa72af5-ef65-470f-a4a9-1ff701351e36 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:00.972572 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:00.972917 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:00.973395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] PATH_INFO: `/v3/projects/d12806b9ad9546fcb32a56e76ced516f` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:00.974736 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d12806b9ad9546fcb32a56e76ced516f)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:00.977090 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.012976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 904/1780] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:00 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:01.014019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa72af5-ef65-470f-a4a9-1ff701351e36 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.069858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd12806b9ad9546fcb32a56e76ced516f', 'request_id': 'req-6fa2b22f-8322-48c7-958c-af2298b6d074'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:01.077519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd12806b9ad9546fcb32a56e76ced516f', 'request_id': 'req-6fa2b22f-8322-48c7-958c-af2298b6d074'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:01.083857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd12806b9ad9546fcb32a56e76ced516f', 'request_id': 'req-6fa2b22f-8322-48c7-958c-af2298b6d074'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:01.118442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] The token cache is being invalidate because project d12806b9ad9546fcb32a56e76ced516f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:01.118640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:01.122825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa72af5-ef65-470f-a4a9-1ff701351e36 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:01.123251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:01.140887 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 877/1781] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:00 2022] DELETE /identity/v3/projects/d12806b9ad9546fcb32a56e76ced516f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:01.142531 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.165221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:01.234738 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c42d0-6ef7-43a8-926a-736f0f42e42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:01.235595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c42d0-6ef7-43a8-926a-736f0f42e42e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.235948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c42d0-6ef7-43a8-926a-736f0f42e42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.236311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c42d0-6ef7-43a8-926a-736f0f42e42e admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.237987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c42d0-6ef7-43a8-926a-736f0f42e42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:01.239893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c42d0-6ef7-43a8-926a-736f0f42e42e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.256610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 905/1782] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:01.256610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dec81e-f9c9-45fd-bfbc-d9e7fba17f55 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.256610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dec81e-f9c9-45fd-bfbc-d9e7fba17f55 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.256610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dec81e-f9c9-45fd-bfbc-d9e7fba17f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.266727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2b22f-8322-48c7-958c-af2298b6d074 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:01.266913 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:01.273494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf8d8a-5ad5-4352-916d-9b5e018cd684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:01.274649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf8d8a-5ad5-4352-916d-9b5e018cd684 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.275137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf8d8a-5ad5-4352-916d-9b5e018cd684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.275604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf8d8a-5ad5-4352-916d-9b5e018cd684 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.276934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf8d8a-5ad5-4352-916d-9b5e018cd684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:01.279385 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf8d8a-5ad5-4352-916d-9b5e018cd684 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.290176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-46dec81e-f9c9-45fd-bfbc-d9e7fba17f55 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:01.293105 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-46dec81e-f9c9-45fd-bfbc-d9e7fba17f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:01.310344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 906/1783] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:01.311852 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dec81e-f9c9-45fd-bfbc-d9e7fba17f55 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.313390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-46dec81e-f9c9-45fd-bfbc-d9e7fba17f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:01.330071 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 878/1784] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:01 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:01.331620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21500fa9-b61c-4931-b401-5b1523a0f3b3 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.332045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21500fa9-b61c-4931-b401-5b1523a0f3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.332608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21500fa9-b61c-4931-b401-5b1523a0f3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.337635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dec81e-f9c9-45fd-bfbc-d9e7fba17f55 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:01.344175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bb19d-5713-4e8c-899d-66fb26da87b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:01.345073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bb19d-5713-4e8c-899d-66fb26da87b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.345469 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bb19d-5713-4e8c-899d-66fb26da87b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.345757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bb19d-5713-4e8c-899d-66fb26da87b7 admin admin] PATH_INFO: `/v3/projects/7f0d8bbf9e704c1ba97b69c6529f43f7/users/6b4d67ffa91a4cad977e86e032abc6ec/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.365473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-21500fa9-b61c-4931-b401-5b1523a0f3b3 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:01.367005 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-21500fa9-b61c-4931-b401-5b1523a0f3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:01.367272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bb19d-5713-4e8c-899d-66fb26da87b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0d8bbf9e704c1ba97b69c6529f43f7, user_id=6b4d67ffa91a4cad977e86e032abc6ec, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:01.372215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bb19d-5713-4e8c-899d-66fb26da87b7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.383940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 879/1785] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:01.384817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21500fa9-b61c-4931-b401-5b1523a0f3b3 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.396271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21500fa9-b61c-4931-b401-5b1523a0f3b3 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:01.397231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 907/1786] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:01 2022] PUT /identity/v3/projects/7f0d8bbf9e704c1ba97b69c6529f43f7/users/6b4d67ffa91a4cad977e86e032abc6ec/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:01.400693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bb19d-5713-4e8c-899d-66fb26da87b7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.402433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:01.478317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679d2a5-27d1-4f5e-aa35-ccaee8acb727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:01.479143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679d2a5-27d1-4f5e-aa35-ccaee8acb727 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.479431 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679d2a5-27d1-4f5e-aa35-ccaee8acb727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.479716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679d2a5-27d1-4f5e-aa35-ccaee8acb727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.480722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679d2a5-27d1-4f5e-aa35-ccaee8acb727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:01.482959 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679d2a5-27d1-4f5e-aa35-ccaee8acb727 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.503696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bb19d-5713-4e8c-899d-66fb26da87b7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:01.504167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 880/1787] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:01.505341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679d2a5-27d1-4f5e-aa35-ccaee8acb727 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.508528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf27dce-6b57-41b4-a5e1-5cd2ff5c7e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:01.509210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf27dce-6b57-41b4-a5e1-5cd2ff5c7e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.509496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf27dce-6b57-41b4-a5e1-5cd2ff5c7e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.512558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf27dce-6b57-41b4-a5e1-5cd2ff5c7e35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.518977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf27dce-6b57-41b4-a5e1-5cd2ff5c7e35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:01.519599 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf27dce-6b57-41b4-a5e1-5cd2ff5c7e35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:01.520200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679d2a5-27d1-4f5e-aa35-ccaee8acb727 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:01.523655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf27dce-6b57-41b4-a5e1-5cd2ff5c7e35 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.525211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf94c01-8426-4a7a-b06b-16454327a6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:01.525798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf94c01-8426-4a7a-b06b-16454327a6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.526112 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf94c01-8426-4a7a-b06b-16454327a6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.526393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf94c01-8426-4a7a-b06b-16454327a6b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.527131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf94c01-8426-4a7a-b06b-16454327a6b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:01.528937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf94c01-8426-4a7a-b06b-16454327a6b2 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.544559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 908/1788] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:40:01.546826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf27dce-6b57-41b4-a5e1-5cd2ff5c7e35 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.549332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 881/1789] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:01.550296 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf94c01-8426-4a7a-b06b-16454327a6b2 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.569315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf27dce-6b57-41b4-a5e1-5cd2ff5c7e35 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:01.572790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf94c01-8426-4a7a-b06b-16454327a6b2 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:01.578977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81745a9-e274-40b8-bd8d-c441db3a9ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:01.579795 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81745a9-e274-40b8-bd8d-c441db3a9ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.580081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81745a9-e274-40b8-bd8d-c441db3a9ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.580356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81745a9-e274-40b8-bd8d-c441db3a9ed4 admin admin] PATH_INFO: `/v3/projects/68ae482fad824961852e91f21e7c13a2/users/6fe5a0a5e23d4778bf35ad04dbb7cf6b/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.585337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e7235-8fe0-4f90-9a59-261d41843e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:01.586321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e7235-8fe0-4f90-9a59-261d41843e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.586608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e7235-8fe0-4f90-9a59-261d41843e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.586884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e7235-8fe0-4f90-9a59-261d41843e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.587593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e7235-8fe0-4f90-9a59-261d41843e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:01.589490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e7235-8fe0-4f90-9a59-261d41843e45 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.617871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81745a9-e274-40b8-bd8d-c441db3a9ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ae482fad824961852e91f21e7c13a2, user_id=6fe5a0a5e23d4778bf35ad04dbb7cf6b, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:01.621105 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81745a9-e274-40b8-bd8d-c441db3a9ed4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.648030 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 909/1790] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:01 2022] PUT /identity/v3/projects/68ae482fad824961852e91f21e7c13a2/users/6fe5a0a5e23d4778bf35ad04dbb7cf6b/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:01.648679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81745a9-e274-40b8-bd8d-c441db3a9ed4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.659407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 882/1791] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:01 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:01.661310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e7235-8fe0-4f90-9a59-261d41843e45 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.800683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e7235-8fe0-4f90-9a59-261d41843e45 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:01.807767 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81745a9-e274-40b8-bd8d-c441db3a9ed4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:01.812427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e51f39-98b2-4136-917e-2cff6c05baef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:01.813121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e51f39-98b2-4136-917e-2cff6c05baef admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.813403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e51f39-98b2-4136-917e-2cff6c05baef admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.813678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e51f39-98b2-4136-917e-2cff6c05baef admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.815084 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e51f39-98b2-4136-917e-2cff6c05baef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:01.817895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e51f39-98b2-4136-917e-2cff6c05baef admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.821018 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99736c4f-2b94-42b4-acc1-1922afddbe5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:01.824181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99736c4f-2b94-42b4-acc1-1922afddbe5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.824748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99736c4f-2b94-42b4-acc1-1922afddbe5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.825232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99736c4f-2b94-42b4-acc1-1922afddbe5c admin admin] PATH_INFO: `/v3/projects/7f0d8bbf9e704c1ba97b69c6529f43f7/users/6b4d67ffa91a4cad977e86e032abc6ec/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.832403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99736c4f-2b94-42b4-acc1-1922afddbe5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0d8bbf9e704c1ba97b69c6529f43f7, user_id=6b4d67ffa91a4cad977e86e032abc6ec, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:01.835853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99736c4f-2b94-42b4-acc1-1922afddbe5c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.839919 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 883/1792] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:01.840878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e51f39-98b2-4136-917e-2cff6c05baef admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.860887 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 910/1793] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:01 2022] PUT /identity/v3/projects/7f0d8bbf9e704c1ba97b69c6529f43f7/users/6b4d67ffa91a4cad977e86e032abc6ec/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:01.861844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99736c4f-2b94-42b4-acc1-1922afddbe5c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.868222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e51f39-98b2-4136-917e-2cff6c05baef admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:01.917651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:01.956161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99736c4f-2b94-42b4-acc1-1922afddbe5c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:01.957299 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e059295-2a6a-453d-8107-f9fef904ec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:01.957876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e059295-2a6a-453d-8107-f9fef904ec51 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.957876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e059295-2a6a-453d-8107-f9fef904ec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.958208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e059295-2a6a-453d-8107-f9fef904ec51 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.959141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e059295-2a6a-453d-8107-f9fef904ec51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:01.961401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e059295-2a6a-453d-8107-f9fef904ec51 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.967602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974e28d-ee80-4ff1-b09e-e3e6e9d8ae82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:01.969372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974e28d-ee80-4ff1-b09e-e3e6e9d8ae82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.969841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974e28d-ee80-4ff1-b09e-e3e6e9d8ae82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.970507 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974e28d-ee80-4ff1-b09e-e3e6e9d8ae82 admin admin] PATH_INFO: `/v3/projects/68ae482fad824961852e91f21e7c13a2/users/6fe5a0a5e23d4778bf35ad04dbb7cf6b/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:01.974781 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974e28d-ee80-4ff1-b09e-e3e6e9d8ae82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ae482fad824961852e91f21e7c13a2, user_id=6fe5a0a5e23d4778bf35ad04dbb7cf6b, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:01.978323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974e28d-ee80-4ff1-b09e-e3e6e9d8ae82 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:01.985198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:01.993385 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 884/1794] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:01 2022] POST /identity/v3/users => generated 284 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:01.994380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 911/1795] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:01 2022] PUT /identity/v3/projects/68ae482fad824961852e91f21e7c13a2/users/6fe5a0a5e23d4778bf35ad04dbb7cf6b/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:01.995847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6232dba-4c57-4a9e-91ec-9246148befe6 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:01.996506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6232dba-4c57-4a9e-91ec-9246148befe6 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:01.997704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974e28d-ee80-4ff1-b09e-e3e6e9d8ae82 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:01.998829 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6232dba-4c57-4a9e-91ec-9246148befe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:02.062096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f6232dba-4c57-4a9e-91ec-9246148befe6 None None] MFA Rules not processed for user `6b4d67ffa91a4cad977e86e032abc6ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:02.063316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f6232dba-4c57-4a9e-91ec-9246148befe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:02.115953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974e28d-ee80-4ff1-b09e-e3e6e9d8ae82 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:02.122000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9f876-e536-44de-926f-4e03502522ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:02.122180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9f876-e536-44de-926f-4e03502522ed admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:02.122711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9f876-e536-44de-926f-4e03502522ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:02.122862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9f876-e536-44de-926f-4e03502522ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:02.123847 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9f876-e536-44de-926f-4e03502522ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:02.125587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9f876-e536-44de-926f-4e03502522ed admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:02.162876 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 912/1796] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:02.164401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2badbc1-3126-4de5-935c-20c664f56bb7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:02.164690 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2badbc1-3126-4de5-935c-20c664f56bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:02.165118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2badbc1-3126-4de5-935c-20c664f56bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:02.207124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 885/1797] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:01 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:02.221158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6232dba-4c57-4a9e-91ec-9246148befe6 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:02.240647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b2badbc1-3126-4de5-935c-20c664f56bb7 None None] MFA Rules not processed for user `6fe5a0a5e23d4778bf35ad04dbb7cf6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:02.241875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b2badbc1-3126-4de5-935c-20c664f56bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:02.254871 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6232dba-4c57-4a9e-91ec-9246148befe6 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:02.268791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e501e9-65e4-47cd-a038-2828f0866c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:02.269218 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e501e9-65e4-47cd-a038-2828f0866c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:02.269539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e501e9-65e4-47cd-a038-2828f0866c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:02.269815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e501e9-65e4-47cd-a038-2828f0866c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:02.270623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e501e9-65e4-47cd-a038-2828f0866c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:02.272454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e501e9-65e4-47cd-a038-2828f0866c76 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:02.300691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 886/1798] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:02.302908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e501e9-65e4-47cd-a038-2828f0866c76 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:02.307601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-24e501e9-65e4-47cd-a038-2828f0866c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:02.408072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 913/1799] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:02.409498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2badbc1-3126-4de5-935c-20c664f56bb7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:02.409634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e501e9-65e4-47cd-a038-2828f0866c76 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:02.413675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fba795-2a2b-4035-b4bd-c4a34ef6b51f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:02.414386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fba795-2a2b-4035-b4bd-c4a34ef6b51f service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:02.414666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fba795-2a2b-4035-b4bd-c4a34ef6b51f service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:02.414948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fba795-2a2b-4035-b4bd-c4a34ef6b51f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:02.424802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2badbc1-3126-4de5-935c-20c664f56bb7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:02.428589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fba795-2a2b-4035-b4bd-c4a34ef6b51f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:02.430098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fba795-2a2b-4035-b4bd-c4a34ef6b51f service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:02.440725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142263f5-efc0-42b5-9691-695b361548ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:02.441747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142263f5-efc0-42b5-9691-695b361548ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:02.442179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142263f5-efc0-42b5-9691-695b361548ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:02.442544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142263f5-efc0-42b5-9691-695b361548ac admin admin] PATH_INFO: `/v3/projects/e00decd887e94aecbd4fdfd7203aa56e/users/1a801e382e024dfeba755df8c4d4ac58/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:02.444266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 887/1800] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:02 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:02.445384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fba795-2a2b-4035-b4bd-c4a34ef6b51f service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:02.447789 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-17fba795-2a2b-4035-b4bd-c4a34ef6b51f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:02.462295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142263f5-efc0-42b5-9691-695b361548ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e00decd887e94aecbd4fdfd7203aa56e, user_id=1a801e382e024dfeba755df8c4d4ac58, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:02.462547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fba795-2a2b-4035-b4bd-c4a34ef6b51f service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:02.466585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142263f5-efc0-42b5-9691-695b361548ac admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:02.469280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5656c-f38d-4054-b4a4-0ee8b5fae84b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:02.469510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5656c-f38d-4054-b4a4-0ee8b5fae84b service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:02.469890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5656c-f38d-4054-b4a4-0ee8b5fae84b service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:02.470314 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5656c-f38d-4054-b4a4-0ee8b5fae84b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:02.473028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-16a5656c-f38d-4054-b4a4-0ee8b5fae84b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:02.486087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 914/1801] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:02 2022] PUT /identity/v3/projects/e00decd887e94aecbd4fdfd7203aa56e/users/1a801e382e024dfeba755df8c4d4ac58/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:02.498077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10413ff-bc4a-4b3e-af6c-54b4156c5a80 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:02.499502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10413ff-bc4a-4b3e-af6c-54b4156c5a80 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:02.499954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10413ff-bc4a-4b3e-af6c-54b4156c5a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:02.545784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c10413ff-bc4a-4b3e-af6c-54b4156c5a80 None None] MFA Rules not processed for user `1a801e382e024dfeba755df8c4d4ac58`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:02.547936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c10413ff-bc4a-4b3e-af6c-54b4156c5a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:02.551402 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5656c-f38d-4054-b4a4-0ee8b5fae84b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:02.553347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5656c-f38d-4054-b4a4-0ee8b5fae84b service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:02.564699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:02.569481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 888/1802] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:02.579790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5656c-f38d-4054-b4a4-0ee8b5fae84b service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:02.595002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5656c-f38d-4054-b4a4-0ee8b5fae84b service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:02.614572 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:02.673495 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65624e3-0905-41f2-b5f0-4e36fd006e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:02.674637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65624e3-0905-41f2-b5f0-4e36fd006e30 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:02.674991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65624e3-0905-41f2-b5f0-4e36fd006e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:02.675394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65624e3-0905-41f2-b5f0-4e36fd006e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:02.678044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c65624e3-0905-41f2-b5f0-4e36fd006e30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:02.695902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 915/1803] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:02.697939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10413ff-bc4a-4b3e-af6c-54b4156c5a80 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:02.782246 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65624e3-0905-41f2-b5f0-4e36fd006e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:02.784562 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65624e3-0905-41f2-b5f0-4e36fd006e30 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:02.801985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10413ff-bc4a-4b3e-af6c-54b4156c5a80 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:02.817072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 889/1804] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:02.818577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c5fe9-30a7-4c6a-8d69-803b45672ea9 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:02.818940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c5fe9-30a7-4c6a-8d69-803b45672ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:02.819401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c5fe9-30a7-4c6a-8d69-803b45672ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:02.821685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22b823-8454-449a-b3a6-5431396ab003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:02.822994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22b823-8454-449a-b3a6-5431396ab003 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:02.823277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22b823-8454-449a-b3a6-5431396ab003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:02.823656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22b823-8454-449a-b3a6-5431396ab003 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:02.824407 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22b823-8454-449a-b3a6-5431396ab003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:02.827875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22b823-8454-449a-b3a6-5431396ab003 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:02.856832 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-716c5fe9-30a7-4c6a-8d69-803b45672ea9 None None] MFA Rules not processed for user `6f611c60491941ca8a0200212ab0c7a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:02.858113 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-716c5fe9-30a7-4c6a-8d69-803b45672ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:02.922378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 916/1805] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:02 2022] POST /identity/v3/projects => generated 369 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:02.923968 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22b823-8454-449a-b3a6-5431396ab003 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:03.007796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 890/1806] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:03.094444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22b823-8454-449a-b3a6-5431396ab003 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:03.119802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e1028-a017-4a5f-8e3f-0a5f6982e869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:03.120691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e1028-a017-4a5f-8e3f-0a5f6982e869 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:03.121079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e1028-a017-4a5f-8e3f-0a5f6982e869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:03.121504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e1028-a017-4a5f-8e3f-0a5f6982e869 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:03.122364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e1028-a017-4a5f-8e3f-0a5f6982e869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:03.128300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e1028-a017-4a5f-8e3f-0a5f6982e869 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:03.214944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 917/1807] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:02 2022] POST /identity/v3/users => generated 291 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:03.216448 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c5fe9-30a7-4c6a-8d69-803b45672ea9 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:03.245574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c5fe9-30a7-4c6a-8d69-803b45672ea9 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:03.257668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813fe26-9f30-48b7-8b43-14a4fadce2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:03.262607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813fe26-9f30-48b7-8b43-14a4fadce2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:03.263257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813fe26-9f30-48b7-8b43-14a4fadce2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:03.266764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813fe26-9f30-48b7-8b43-14a4fadce2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:03.269096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813fe26-9f30-48b7-8b43-14a4fadce2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:03.274960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813fe26-9f30-48b7-8b43-14a4fadce2c0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:03.307769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 891/1808] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:03.312909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e1028-a017-4a5f-8e3f-0a5f6982e869 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:03.324077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e1028-a017-4a5f-8e3f-0a5f6982e869 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:03.346627 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e49aa-07ea-4164-adbf-13eb75ab1699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:03.347859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e49aa-07ea-4164-adbf-13eb75ab1699 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:03.348150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e49aa-07ea-4164-adbf-13eb75ab1699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:03.348464 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e49aa-07ea-4164-adbf-13eb75ab1699 admin admin] PATH_INFO: `/v3/projects/4b75343619e54573942614c826f04600/users/84964d859f244a978a08bbfa34f72921/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:03.399258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e49aa-07ea-4164-adbf-13eb75ab1699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b75343619e54573942614c826f04600, user_id=84964d859f244a978a08bbfa34f72921, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:03.402022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e49aa-07ea-4164-adbf-13eb75ab1699 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:03.430809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 918/1809] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:03 2022] PUT /identity/v3/projects/4b75343619e54573942614c826f04600/users/84964d859f244a978a08bbfa34f72921/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:03.435027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813fe26-9f30-48b7-8b43-14a4fadce2c0 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:03.468803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813fe26-9f30-48b7-8b43-14a4fadce2c0 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:03.565665 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9e2fe-111c-4cec-ae0f-50ebdaf69c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:03.567687 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9e2fe-111c-4cec-ae0f-50ebdaf69c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:03.568185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9e2fe-111c-4cec-ae0f-50ebdaf69c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:03.569425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9e2fe-111c-4cec-ae0f-50ebdaf69c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:03.570482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9e2fe-111c-4cec-ae0f-50ebdaf69c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:03.573247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9e2fe-111c-4cec-ae0f-50ebdaf69c59 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:03.580430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:03.590500 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e49aa-07ea-4164-adbf-13eb75ab1699 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:03.597835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 892/1810] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:03.606509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e49aa-07ea-4164-adbf-13eb75ab1699 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:03.613551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6a0b3-5bc5-458b-bcb6-ba7a5f363946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:03.619123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6a0b3-5bc5-458b-bcb6-ba7a5f363946 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:03.619720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6a0b3-5bc5-458b-bcb6-ba7a5f363946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:03.620196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6a0b3-5bc5-458b-bcb6-ba7a5f363946 admin admin] PATH_INFO: `/v3/projects/4b75343619e54573942614c826f04600/users/84964d859f244a978a08bbfa34f72921/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:03.623693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6a0b3-5bc5-458b-bcb6-ba7a5f363946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b75343619e54573942614c826f04600, user_id=84964d859f244a978a08bbfa34f72921, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:03.628144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6a0b3-5bc5-458b-bcb6-ba7a5f363946 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:03.631002 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:03.645363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 919/1811] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:03 2022] PUT /identity/v3/projects/4b75343619e54573942614c826f04600/users/84964d859f244a978a08bbfa34f72921/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:03.657639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6b1fd-eb14-4d93-a8f1-921c9134ecae None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:03.657903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6b1fd-eb14-4d93-a8f1-921c9134ecae None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:03.658305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6b1fd-eb14-4d93-a8f1-921c9134ecae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:03.677158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6a0b3-5bc5-458b-bcb6-ba7a5f363946 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:03.679138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4ce6a0b3-5bc5-458b-bcb6-ba7a5f363946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:03.704552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-97d6b1fd-eb14-4d93-a8f1-921c9134ecae None None] MFA Rules not processed for user `84964d859f244a978a08bbfa34f72921`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:03.706158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-97d6b1fd-eb14-4d93-a8f1-921c9134ecae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:03.779447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6a0b3-5bc5-458b-bcb6-ba7a5f363946 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:03.787087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:03.788957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:03.789320 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:03.789677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] PATH_INFO: `/v3/users/4de9b84f323b4cddb99640a7f9625380` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:03.792885 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de9b84f323b4cddb99640a7f9625380)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:03.795899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:03.816537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 893/1812] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:03.832493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6b1fd-eb14-4d93-a8f1-921c9134ecae None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:03.887434 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de9b84f323b4cddb99640a7f9625380'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:03.905456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6b1fd-eb14-4d93-a8f1-921c9134ecae None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:03.906716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de9b84f323b4cddb99640a7f9625380'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:03.909743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17c900-43b6-4eff-84ed-be25ed8506b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:03.910620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17c900-43b6-4eff-84ed-be25ed8506b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:03.911189 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17c900-43b6-4eff-84ed-be25ed8506b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:03.911573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17c900-43b6-4eff-84ed-be25ed8506b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:03.914122 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ba17c900-43b6-4eff-84ed-be25ed8506b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:03.914577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de9b84f323b4cddb99640a7f9625380'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:03.923559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de9b84f323b4cddb99640a7f9625380'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:03.924623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de9b84f323b4cddb99640a7f9625380'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:03.929513 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17c900-43b6-4eff-84ed-be25ed8506b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:03.931525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17c900-43b6-4eff-84ed-be25ed8506b9 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:03.944257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 894/1813] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:03 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:40:03.947038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 920/1814] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:03 2022] DELETE /identity/v3/users/4de9b84f323b4cddb99640a7f9625380 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:03.952147 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3201a-b5a4-4a98-bc4d-753418d4c7f4 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:03.952508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3201a-b5a4-4a98-bc4d-753418d4c7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:03.952854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3201a-b5a4-4a98-bc4d-753418d4c7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:03.977794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f3a3201a-b5a4-4a98-bc4d-753418d4c7f4 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:03.979171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f3a3201a-b5a4-4a98-bc4d-753418d4c7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:04.000973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:04.003006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:04.080488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a26be3-b56b-4e73-9f4d-8937d57c67b2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:04.085869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535a609-7b64-4989-af77-8a3294d388db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:04.086905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535a609-7b64-4989-af77-8a3294d388db service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:04.087298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535a609-7b64-4989-af77-8a3294d388db service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:04.087680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535a609-7b64-4989-af77-8a3294d388db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:04.090334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d535a609-7b64-4989-af77-8a3294d388db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:04.092654 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 895/1815] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:04.101595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3201a-b5a4-4a98-bc4d-753418d4c7f4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:04.103228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f3a3201a-b5a4-4a98-bc4d-753418d4c7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:04.132094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:04.138227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:04.173171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535a609-7b64-4989-af77-8a3294d388db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:04.175098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535a609-7b64-4989-af77-8a3294d388db service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:04.182553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3201a-b5a4-4a98-bc4d-753418d4c7f4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:04.188161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 921/1816] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:04.209421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5ee1d-a5a4-4976-a85d-7872584716bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:04.210132 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5ee1d-a5a4-4976-a85d-7872584716bb service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:04.212024 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5ee1d-a5a4-4976-a85d-7872584716bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:04.212324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5ee1d-a5a4-4976-a85d-7872584716bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:04.226927 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5ee1d-a5a4-4976-a85d-7872584716bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:04.228768 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5ee1d-a5a4-4976-a85d-7872584716bb service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:04.254011 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 896/1817] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:40:04.603646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535a609-7b64-4989-af77-8a3294d388db service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:04.605215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d535a609-7b64-4989-af77-8a3294d388db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:04.667975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535a609-7b64-4989-af77-8a3294d388db service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:04.686778 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:04.687679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:04.687963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:04.688245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] PATH_INFO: `/v3/projects/83d5ea686e98454bbe714611c77c69e6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:04.690484 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=83d5ea686e98454bbe714611c77c69e6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:04.693681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:04.779950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83d5ea686e98454bbe714611c77c69e6', 'request_id': 'req-ae7c039b-2298-42fa-9803-974858bc2fec'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:04.789010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83d5ea686e98454bbe714611c77c69e6', 'request_id': 'req-ae7c039b-2298-42fa-9803-974858bc2fec'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:04.797756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83d5ea686e98454bbe714611c77c69e6', 'request_id': 'req-ae7c039b-2298-42fa-9803-974858bc2fec'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:04.811982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] The token cache is being invalidate because project 83d5ea686e98454bbe714611c77c69e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:04.812372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:04.813268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:04.814915 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 922/1818] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:04 2022] DELETE /identity/v3/projects/83d5ea686e98454bbe714611c77c69e6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:04.819153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5ee1d-a5a4-4976-a85d-7872584716bb service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:04.829775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5ee1d-a5a4-4976-a85d-7872584716bb service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:04.905126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3967ff6-e5fa-46cf-944a-7afb7ba6afcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:04.906465 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3967ff6-e5fa-46cf-944a-7afb7ba6afcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:04.907174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3967ff6-e5fa-46cf-944a-7afb7ba6afcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:04.907600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3967ff6-e5fa-46cf-944a-7afb7ba6afcc admin admin] PATH_INFO: `/v3/users/9764d3a3cc6e4f22a3a2954a31070311` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:04.910260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3967ff6-e5fa-46cf-944a-7afb7ba6afcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9764d3a3cc6e4f22a3a2954a31070311)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:04.913340 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3967ff6-e5fa-46cf-944a-7afb7ba6afcc admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:05.058351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a3967ff6-e5fa-46cf-944a-7afb7ba6afcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9764d3a3cc6e4f22a3a2954a31070311'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:05.059164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a3967ff6-e5fa-46cf-944a-7afb7ba6afcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9764d3a3cc6e4f22a3a2954a31070311'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:05.059830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a3967ff6-e5fa-46cf-944a-7afb7ba6afcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9764d3a3cc6e4f22a3a2954a31070311'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:05.068869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a3967ff6-e5fa-46cf-944a-7afb7ba6afcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9764d3a3cc6e4f22a3a2954a31070311'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:05.079440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a3967ff6-e5fa-46cf-944a-7afb7ba6afcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9764d3a3cc6e4f22a3a2954a31070311'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:05.102792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 897/1819] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:04 2022] DELETE /identity/v3/users/9764d3a3cc6e4f22a3a2954a31070311 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:05.455796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:05.457251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:05.472639 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c039b-2298-42fa-9803-974858bc2fec admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:05.533589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b914041-a7ce-4352-b6c3-862ffc27e6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:05.534503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b914041-a7ce-4352-b6c3-862ffc27e6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:05.534790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b914041-a7ce-4352-b6c3-862ffc27e6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:05.535068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b914041-a7ce-4352-b6c3-862ffc27e6f3 admin admin] PATH_INFO: `/v3/projects/9d9913d9fc854aca86e525983408ab7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:05.536246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b914041-a7ce-4352-b6c3-862ffc27e6f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d9913d9fc854aca86e525983408ab7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:05.538226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b914041-a7ce-4352-b6c3-862ffc27e6f3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:05.583209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7b914041-a7ce-4352-b6c3-862ffc27e6f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d9913d9fc854aca86e525983408ab7e', 'request_id': 'req-7b914041-a7ce-4352-b6c3-862ffc27e6f3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:05.585637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:05.590762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7b914041-a7ce-4352-b6c3-862ffc27e6f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d9913d9fc854aca86e525983408ab7e', 'request_id': 'req-7b914041-a7ce-4352-b6c3-862ffc27e6f3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:05.598290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7b914041-a7ce-4352-b6c3-862ffc27e6f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d9913d9fc854aca86e525983408ab7e', 'request_id': 'req-7b914041-a7ce-4352-b6c3-862ffc27e6f3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:05.610315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7b914041-a7ce-4352-b6c3-862ffc27e6f3 admin admin] The token cache is being invalidate because project 9d9913d9fc854aca86e525983408ab7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:05.610605 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7b914041-a7ce-4352-b6c3-862ffc27e6f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:05.611258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7b914041-a7ce-4352-b6c3-862ffc27e6f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:05.612823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 923/1820] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:05 2022] DELETE /identity/v3/projects/9d9913d9fc854aca86e525983408ab7e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:05.617639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b58f09-a94b-4d93-92b8-6ab143979053 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:05.618064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b58f09-a94b-4d93-92b8-6ab143979053 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:05.618574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b58f09-a94b-4d93-92b8-6ab143979053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:05.638511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-52b58f09-a94b-4d93-92b8-6ab143979053 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:05.639706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-52b58f09-a94b-4d93-92b8-6ab143979053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:05.683330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:05.700681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 898/1821] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:05.708442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bab90f-853f-4846-afa3-72493d4cf2db None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:05.708735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bab90f-853f-4846-afa3-72493d4cf2db None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:05.709019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bab90f-853f-4846-afa3-72493d4cf2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:05.728383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a3bab90f-853f-4846-afa3-72493d4cf2db None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:05.729581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a3bab90f-853f-4846-afa3-72493d4cf2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:05.738404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 924/1822] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:05.742594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b58f09-a94b-4d93-92b8-6ab143979053 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:05.753088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b58f09-a94b-4d93-92b8-6ab143979053 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:05.757759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a077b2-f906-40ef-a87b-72b38bf2fc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:05.758454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a077b2-f906-40ef-a87b-72b38bf2fc9d admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:05.758741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a077b2-f906-40ef-a87b-72b38bf2fc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:05.759020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a077b2-f906-40ef-a87b-72b38bf2fc9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:05.759790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a077b2-f906-40ef-a87b-72b38bf2fc9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:05.760103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a077b2-f906-40ef-a87b-72b38bf2fc9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:05.761721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a077b2-f906-40ef-a87b-72b38bf2fc9d admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:05.773498 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 899/1823] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:40:05.777520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bab90f-853f-4846-afa3-72493d4cf2db None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:05.787485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bab90f-853f-4846-afa3-72493d4cf2db None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:05.792135 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc143c39-dc46-461b-86a8-edba3c8b5d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:05.792796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc143c39-dc46-461b-86a8-edba3c8b5d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:05.793077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc143c39-dc46-461b-86a8-edba3c8b5d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:05.793354 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc143c39-dc46-461b-86a8-edba3c8b5d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:05.794081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc143c39-dc46-461b-86a8-edba3c8b5d6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:05.796535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc143c39-dc46-461b-86a8-edba3c8b5d6b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:05.825182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 925/1824] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:05 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:05.829342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a077b2-f906-40ef-a87b-72b38bf2fc9d admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:05.839847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a077b2-f906-40ef-a87b-72b38bf2fc9d admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:05.902321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3889a43-24ec-4a5f-a44c-0157680612cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:05.903388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3889a43-24ec-4a5f-a44c-0157680612cf admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:05.903676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3889a43-24ec-4a5f-a44c-0157680612cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:05.903958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3889a43-24ec-4a5f-a44c-0157680612cf admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:05.905224 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3889a43-24ec-4a5f-a44c-0157680612cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:05.907946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3889a43-24ec-4a5f-a44c-0157680612cf admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:05.937198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 900/1825] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:05 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:05.938412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc143c39-dc46-461b-86a8-edba3c8b5d6b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:05.949055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc143c39-dc46-461b-86a8-edba3c8b5d6b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:05.957255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de883a2-166b-4671-9150-a27dc790908b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:05.957937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de883a2-166b-4671-9150-a27dc790908b admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:05.958238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de883a2-166b-4671-9150-a27dc790908b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:05.958514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de883a2-166b-4671-9150-a27dc790908b admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:05.959238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de883a2-166b-4671-9150-a27dc790908b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:05.960937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de883a2-166b-4671-9150-a27dc790908b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:05.974352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 926/1826] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:05.975744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3889a43-24ec-4a5f-a44c-0157680612cf admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:05.987201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3889a43-24ec-4a5f-a44c-0157680612cf admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:05.992821 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39855035-3cfe-4e08-8649-ca5ed94d3f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:05.993652 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39855035-3cfe-4e08-8649-ca5ed94d3f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:05.994066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39855035-3cfe-4e08-8649-ca5ed94d3f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:05.994435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39855035-3cfe-4e08-8649-ca5ed94d3f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:05.995274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39855035-3cfe-4e08-8649-ca5ed94d3f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:05.997002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39855035-3cfe-4e08-8649-ca5ed94d3f63 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:06.009963 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 901/1827] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:06.014977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de883a2-166b-4671-9150-a27dc790908b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:06.025709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de883a2-166b-4671-9150-a27dc790908b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:06.030753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff592002-d8ea-44b2-82fe-3a7f441972d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:06.031744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff592002-d8ea-44b2-82fe-3a7f441972d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:06.032126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff592002-d8ea-44b2-82fe-3a7f441972d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:06.032468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff592002-d8ea-44b2-82fe-3a7f441972d6 admin admin] PATH_INFO: `/v3/projects/2a4bed0a77f34ddaa655b9c17f29825f/users/cb0b87ea736144f0b41c020a6c0a0eef/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:06.050049 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff592002-d8ea-44b2-82fe-3a7f441972d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4bed0a77f34ddaa655b9c17f29825f, user_id=cb0b87ea736144f0b41c020a6c0a0eef, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:06.052735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff592002-d8ea-44b2-82fe-3a7f441972d6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:06.067970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 927/1828] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:06 2022] PUT /identity/v3/projects/2a4bed0a77f34ddaa655b9c17f29825f/users/cb0b87ea736144f0b41c020a6c0a0eef/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:06.079019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbe300-7795-4f3f-94a9-22a807db0ad4 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:06.079340 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbe300-7795-4f3f-94a9-22a807db0ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:06.079788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbe300-7795-4f3f-94a9-22a807db0ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:06.110510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-95cbe300-7795-4f3f-94a9-22a807db0ad4 None None] MFA Rules not processed for user `cb0b87ea736144f0b41c020a6c0a0eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:06.111778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-95cbe300-7795-4f3f-94a9-22a807db0ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:06.220169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 902/1829] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:06.224962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff592002-d8ea-44b2-82fe-3a7f441972d6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:06.339345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff592002-d8ea-44b2-82fe-3a7f441972d6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:06.345481 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacabc3c-08e1-48c1-81f3-613a013c4b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:06.346334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacabc3c-08e1-48c1-81f3-613a013c4b66 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:06.346752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacabc3c-08e1-48c1-81f3-613a013c4b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:06.347194 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacabc3c-08e1-48c1-81f3-613a013c4b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:06.348060 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacabc3c-08e1-48c1-81f3-613a013c4b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:06.350261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacabc3c-08e1-48c1-81f3-613a013c4b66 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:06.398070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 928/1830] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:06 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:06.402280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbe300-7795-4f3f-94a9-22a807db0ad4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:06.414214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbe300-7795-4f3f-94a9-22a807db0ad4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:06.465204 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:06.485747 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91c49e-8ce1-45ae-b5ca-a17c92037750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:06.486479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91c49e-8ce1-45ae-b5ca-a17c92037750 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:06.486784 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91c49e-8ce1-45ae-b5ca-a17c92037750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:06.487061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91c49e-8ce1-45ae-b5ca-a17c92037750 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:06.487771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91c49e-8ce1-45ae-b5ca-a17c92037750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:06.489705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91c49e-8ce1-45ae-b5ca-a17c92037750 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:06.515178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 903/1831] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:06 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:06.519145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacabc3c-08e1-48c1-81f3-613a013c4b66 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:06.532374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacabc3c-08e1-48c1-81f3-613a013c4b66 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:06.537138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ead7b-57cf-490c-a396-a883246921d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:06.537814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ead7b-57cf-490c-a396-a883246921d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:06.538152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ead7b-57cf-490c-a396-a883246921d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:06.538441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ead7b-57cf-490c-a396-a883246921d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:06.539175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ead7b-57cf-490c-a396-a883246921d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:06.540792 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ead7b-57cf-490c-a396-a883246921d9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:06.553242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 929/1832] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:06.557642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91c49e-8ce1-45ae-b5ca-a17c92037750 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:06.567840 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91c49e-8ce1-45ae-b5ca-a17c92037750 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:06.572549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070c9e6-e161-4c41-9bc4-2f95a7be0f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:06.573459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070c9e6-e161-4c41-9bc4-2f95a7be0f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:06.573744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070c9e6-e161-4c41-9bc4-2f95a7be0f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:06.574059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070c9e6-e161-4c41-9bc4-2f95a7be0f4f admin admin] PATH_INFO: `/v3/projects/3e0c6a7972864dcd8e7d992ae196f314/users/56070d690cef4063a026d69d93937537/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:06.591470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070c9e6-e161-4c41-9bc4-2f95a7be0f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0c6a7972864dcd8e7d992ae196f314, user_id=56070d690cef4063a026d69d93937537, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:06.594708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070c9e6-e161-4c41-9bc4-2f95a7be0f4f admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:06.608092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 904/1833] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:06 2022] PUT /identity/v3/projects/3e0c6a7972864dcd8e7d992ae196f314/users/56070d690cef4063a026d69d93937537/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:06.612345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ead7b-57cf-490c-a396-a883246921d9 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:06.622815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ead7b-57cf-490c-a396-a883246921d9 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:06.669362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:06.680575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2907c3c-bde6-4763-aa4b-d3be1e85d56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:06.681237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2907c3c-bde6-4763-aa4b-d3be1e85d56f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:06.681517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2907c3c-bde6-4763-aa4b-d3be1e85d56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:06.681789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2907c3c-bde6-4763-aa4b-d3be1e85d56f admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:06.682536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2907c3c-bde6-4763-aa4b-d3be1e85d56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:06.684116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2907c3c-bde6-4763-aa4b-d3be1e85d56f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:06.693650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 930/1834] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:06.697971 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070c9e6-e161-4c41-9bc4-2f95a7be0f4f admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:06.708195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070c9e6-e161-4c41-9bc4-2f95a7be0f4f admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:06.712810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498793cf-c4b5-45b5-b3cf-868de431d375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:06.713705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498793cf-c4b5-45b5-b3cf-868de431d375 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:06.714016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498793cf-c4b5-45b5-b3cf-868de431d375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:06.714294 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498793cf-c4b5-45b5-b3cf-868de431d375 admin admin] PATH_INFO: `/v3/projects/3e0c6a7972864dcd8e7d992ae196f314/users/56070d690cef4063a026d69d93937537/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:06.716262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498793cf-c4b5-45b5-b3cf-868de431d375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0c6a7972864dcd8e7d992ae196f314, user_id=56070d690cef4063a026d69d93937537, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:06.719449 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498793cf-c4b5-45b5-b3cf-868de431d375 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:06.729542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 905/1835] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:06 2022] PUT /identity/v3/projects/3e0c6a7972864dcd8e7d992ae196f314/users/56070d690cef4063a026d69d93937537/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:06.741529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebe536-771f-41ad-a161-ccb7a7ccf717 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:06.741813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebe536-771f-41ad-a161-ccb7a7ccf717 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:06.742121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebe536-771f-41ad-a161-ccb7a7ccf717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:06.768610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ccebe536-771f-41ad-a161-ccb7a7ccf717 None None] MFA Rules not processed for user `56070d690cef4063a026d69d93937537`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:06.769734 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ccebe536-771f-41ad-a161-ccb7a7ccf717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:06.836280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 931/1836] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:06.847949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498793cf-c4b5-45b5-b3cf-868de431d375 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:06.849699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-498793cf-c4b5-45b5-b3cf-868de431d375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:06.909317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498793cf-c4b5-45b5-b3cf-868de431d375 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:06.912956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e9b77-e641-4034-a26e-bb138e932a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:06.913603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e9b77-e641-4034-a26e-bb138e932a33 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:06.913890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e9b77-e641-4034-a26e-bb138e932a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:06.914200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e9b77-e641-4034-a26e-bb138e932a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:06.923197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e9b77-e641-4034-a26e-bb138e932a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:06.924624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e9b77-e641-4034-a26e-bb138e932a33 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:06.934044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 906/1837] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:07.010365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebe536-771f-41ad-a161-ccb7a7ccf717 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:07.021888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebe536-771f-41ad-a161-ccb7a7ccf717 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:07.026787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69120a69-6a00-489f-968d-5be2c3669fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:07.027454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69120a69-6a00-489f-968d-5be2c3669fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:07.027737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69120a69-6a00-489f-968d-5be2c3669fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:07.028013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69120a69-6a00-489f-968d-5be2c3669fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:07.083868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69120a69-6a00-489f-968d-5be2c3669fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:07.085315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69120a69-6a00-489f-968d-5be2c3669fe5 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:07.094901 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 932/1838] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:07.854152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e9b77-e641-4034-a26e-bb138e932a33 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:07.855574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9e9e9b77-e641-4034-a26e-bb138e932a33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:07.918806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e9b77-e641-4034-a26e-bb138e932a33 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:07.923923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:07.924810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:07.925096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:07.925370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] PATH_INFO: `/v3/users/1a801e382e024dfeba755df8c4d4ac58` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:07.926563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a801e382e024dfeba755df8c4d4ac58)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:07.928691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:07.968495 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:07.974616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a801e382e024dfeba755df8c4d4ac58'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:07.975459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a801e382e024dfeba755df8c4d4ac58'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:07.976107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a801e382e024dfeba755df8c4d4ac58'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:07.983946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a801e382e024dfeba755df8c4d4ac58'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:07.991461 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a801e382e024dfeba755df8c4d4ac58'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:08.009150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 907/1839] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:07 2022] DELETE /identity/v3/users/1a801e382e024dfeba755df8c4d4ac58 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:08.014331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0ebd7-7429-439a-89e5-2de7cba48224 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:08.014863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0ebd7-7429-439a-89e5-2de7cba48224 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:08.014952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0ebd7-7429-439a-89e5-2de7cba48224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:08.034457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5dd0ebd7-7429-439a-89e5-2de7cba48224 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:08.035569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5dd0ebd7-7429-439a-89e5-2de7cba48224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:08.092890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 933/1840] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:08.102554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:08.104015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:08.160274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaad5e-e61d-4376-9ecf-268f2e465db1 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:08.163871 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276e70d-9860-4d8d-9f28-9f14f645e58e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:08.164512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276e70d-9860-4d8d-9f28-9f14f645e58e service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:08.164797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276e70d-9860-4d8d-9f28-9f14f645e58e service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:08.165074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276e70d-9860-4d8d-9f28-9f14f645e58e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:08.174855 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276e70d-9860-4d8d-9f28-9f14f645e58e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:08.176286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276e70d-9860-4d8d-9f28-9f14f645e58e service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:08.185850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 908/1841] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:08.475298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0ebd7-7429-439a-89e5-2de7cba48224 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:08.477025 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5dd0ebd7-7429-439a-89e5-2de7cba48224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:08.490944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0ebd7-7429-439a-89e5-2de7cba48224 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:08.495838 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:08.496715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:08.496997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:08.497273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] PATH_INFO: `/v3/projects/e00decd887e94aecbd4fdfd7203aa56e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:08.498521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e00decd887e94aecbd4fdfd7203aa56e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:08.500454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:08.507544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:08.558276 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e00decd887e94aecbd4fdfd7203aa56e', 'request_id': 'req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:08.568995 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e00decd887e94aecbd4fdfd7203aa56e', 'request_id': 'req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:08.579034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e00decd887e94aecbd4fdfd7203aa56e', 'request_id': 'req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:08.592459 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] The token cache is being invalidate because project e00decd887e94aecbd4fdfd7203aa56e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:08.592851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:08.593701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:08.596202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 934/1842] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:08 2022] DELETE /identity/v3/projects/e00decd887e94aecbd4fdfd7203aa56e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:08.599947 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276e70d-9860-4d8d-9f28-9f14f645e58e service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:08.610956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276e70d-9860-4d8d-9f28-9f14f645e58e service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:08.684261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84870f26-c2eb-4513-a62e-2bb5afad9b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:08.685149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84870f26-c2eb-4513-a62e-2bb5afad9b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:08.685461 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84870f26-c2eb-4513-a62e-2bb5afad9b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:08.685740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84870f26-c2eb-4513-a62e-2bb5afad9b87 admin admin] PATH_INFO: `/v3/users/84964d859f244a978a08bbfa34f72921` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:08.687288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84870f26-c2eb-4513-a62e-2bb5afad9b87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84964d859f244a978a08bbfa34f72921)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:08.694783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84870f26-c2eb-4513-a62e-2bb5afad9b87 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:08.753253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-84870f26-c2eb-4513-a62e-2bb5afad9b87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84964d859f244a978a08bbfa34f72921'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:08.753884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-84870f26-c2eb-4513-a62e-2bb5afad9b87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84964d859f244a978a08bbfa34f72921'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:08.755015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-84870f26-c2eb-4513-a62e-2bb5afad9b87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84964d859f244a978a08bbfa34f72921'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:08.763114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-84870f26-c2eb-4513-a62e-2bb5afad9b87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84964d859f244a978a08bbfa34f72921'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:08.773154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-84870f26-c2eb-4513-a62e-2bb5afad9b87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84964d859f244a978a08bbfa34f72921'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:08.774706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:08.794223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 909/1843] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:08 2022] DELETE /identity/v3/users/84964d859f244a978a08bbfa34f72921 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:09.236878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:09.238551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:09.252973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b7ad4-8b6f-4bff-8b38-15a712468e73 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:09.323064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddad1fb-1f73-48cd-90fe-9c66743cafef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:09.323064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddad1fb-1f73-48cd-90fe-9c66743cafef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:09.323064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddad1fb-1f73-48cd-90fe-9c66743cafef admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:09.323064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddad1fb-1f73-48cd-90fe-9c66743cafef admin admin] PATH_INFO: `/v3/projects/4b75343619e54573942614c826f04600` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:09.323418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddad1fb-1f73-48cd-90fe-9c66743cafef admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b75343619e54573942614c826f04600)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:09.327839 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddad1fb-1f73-48cd-90fe-9c66743cafef admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:09.393539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0ddad1fb-1f73-48cd-90fe-9c66743cafef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b75343619e54573942614c826f04600', 'request_id': 'req-0ddad1fb-1f73-48cd-90fe-9c66743cafef'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:09.403583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0ddad1fb-1f73-48cd-90fe-9c66743cafef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b75343619e54573942614c826f04600', 'request_id': 'req-0ddad1fb-1f73-48cd-90fe-9c66743cafef'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:09.411446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0ddad1fb-1f73-48cd-90fe-9c66743cafef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b75343619e54573942614c826f04600', 'request_id': 'req-0ddad1fb-1f73-48cd-90fe-9c66743cafef'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:09.426077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0ddad1fb-1f73-48cd-90fe-9c66743cafef admin admin] The token cache is being invalidate because project 4b75343619e54573942614c826f04600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:09.426366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0ddad1fb-1f73-48cd-90fe-9c66743cafef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:09.427160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0ddad1fb-1f73-48cd-90fe-9c66743cafef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:09.428955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 935/1844] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:09 2022] DELETE /identity/v3/projects/4b75343619e54573942614c826f04600 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:09.433846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d63b6a-9afe-43f7-88fc-2ffb8f367931 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:09.436315 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d63b6a-9afe-43f7-88fc-2ffb8f367931 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:09.436315 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d63b6a-9afe-43f7-88fc-2ffb8f367931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:09.456245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-93d63b6a-9afe-43f7-88fc-2ffb8f367931 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:09.457615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-93d63b6a-9afe-43f7-88fc-2ffb8f367931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:09.530018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 910/1845] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:09.537233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b020f-f9f9-4e03-9bf3-5308676619e0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:09.537518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b020f-f9f9-4e03-9bf3-5308676619e0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:09.537819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b020f-f9f9-4e03-9bf3-5308676619e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:09.558963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-8f8b020f-f9f9-4e03-9bf3-5308676619e0 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:09.560109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8f8b020f-f9f9-4e03-9bf3-5308676619e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:09.568637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 936/1846] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:09.573119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d63b6a-9afe-43f7-88fc-2ffb8f367931 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:09.583877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d63b6a-9afe-43f7-88fc-2ffb8f367931 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:09.588710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a898-8642-45da-bb7b-90779b36267e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:09.589377 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a898-8642-45da-bb7b-90779b36267e admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:09.589657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a898-8642-45da-bb7b-90779b36267e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:09.589940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a898-8642-45da-bb7b-90779b36267e admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:09.590677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1a898-8642-45da-bb7b-90779b36267e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:09.591095 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1a898-8642-45da-bb7b-90779b36267e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:09.593224 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1a898-8642-45da-bb7b-90779b36267e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:09.615050 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 911/1847] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:40:09.616170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b020f-f9f9-4e03-9bf3-5308676619e0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:09.627478 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b020f-f9f9-4e03-9bf3-5308676619e0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:09.635171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba9823-e21b-4822-b607-afb417b30ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:09.635899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba9823-e21b-4822-b607-afb417b30ece admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:09.636185 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba9823-e21b-4822-b607-afb417b30ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:09.636455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba9823-e21b-4822-b607-afb417b30ece admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:09.637148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba9823-e21b-4822-b607-afb417b30ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:09.639731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba9823-e21b-4822-b607-afb417b30ece admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:09.676221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:09.692986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 937/1848] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:09 2022] POST /identity/v3/projects => generated 387 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:09.696812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a898-8642-45da-bb7b-90779b36267e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:09.715035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a898-8642-45da-bb7b-90779b36267e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:09.827694 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba9823-e21b-4822-b607-afb417b30ece admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:09.830268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c1ba9823-e21b-4822-b607-afb417b30ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:09.845782 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdf1c3-a1e8-4ebf-930f-d991fc08ee6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:09.846684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdf1c3-a1e8-4ebf-930f-d991fc08ee6d admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:09.846978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdf1c3-a1e8-4ebf-930f-d991fc08ee6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:09.847261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdf1c3-a1e8-4ebf-930f-d991fc08ee6d admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:09.847967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdf1c3-a1e8-4ebf-930f-d991fc08ee6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:09.850733 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdf1c3-a1e8-4ebf-930f-d991fc08ee6d admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:09.885275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 912/1849] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:09 2022] POST /identity/v3/users => generated 294 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:09.889487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdf1c3-a1e8-4ebf-930f-d991fc08ee6d admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:09.901246 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdf1c3-a1e8-4ebf-930f-d991fc08ee6d admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:09.901755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba9823-e21b-4822-b607-afb417b30ece admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:09.906236 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6656f-c746-4f67-bf67-1eecaab1387a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:09.906716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d216ce-74d4-4bef-846a-a5f539410a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:09.906883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6656f-c746-4f67-bf67-1eecaab1387a service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:09.907167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6656f-c746-4f67-bf67-1eecaab1387a service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:09.907334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d216ce-74d4-4bef-846a-a5f539410a7e admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:09.907447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6656f-c746-4f67-bf67-1eecaab1387a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:09.907623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d216ce-74d4-4bef-846a-a5f539410a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:09.907903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d216ce-74d4-4bef-846a-a5f539410a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:09.908631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d216ce-74d4-4bef-846a-a5f539410a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:09.909201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f8b6656f-c746-4f67-bf67-1eecaab1387a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:09.910259 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d216ce-74d4-4bef-846a-a5f539410a7e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:09.924633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 913/1850] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:09.925491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d216ce-74d4-4bef-846a-a5f539410a7e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:09.929037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-56d216ce-74d4-4bef-846a-a5f539410a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:09.950676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d216ce-74d4-4bef-846a-a5f539410a7e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:09.957528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2325f6-f776-4e20-9358-b43c2e825bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:09.959127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2325f6-f776-4e20-9358-b43c2e825bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:09.959411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2325f6-f776-4e20-9358-b43c2e825bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:09.959688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2325f6-f776-4e20-9358-b43c2e825bd0 admin admin] PATH_INFO: `/v3/users/493b1466e40a41319e02bc6711b3bb75` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:09.961441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2325f6-f776-4e20-9358-b43c2e825bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=493b1466e40a41319e02bc6711b3bb75)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:09.963884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2325f6-f776-4e20-9358-b43c2e825bd0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:09.995348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6656f-c746-4f67-bf67-1eecaab1387a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:09.998152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6656f-c746-4f67-bf67-1eecaab1387a service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:10.010634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 938/1851] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:40:10.011559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6656f-c746-4f67-bf67-1eecaab1387a service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:10.021097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:10.024502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6656f-c746-4f67-bf67-1eecaab1387a service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:10.032765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39433a-5c82-427d-b723-157255ff08e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:10.034130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39433a-5c82-427d-b723-157255ff08e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:10.034603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39433a-5c82-427d-b723-157255ff08e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:10.035253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39433a-5c82-427d-b723-157255ff08e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:10.035912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39433a-5c82-427d-b723-157255ff08e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:10.036685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9d2325f6-f776-4e20-9358-b43c2e825bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '493b1466e40a41319e02bc6711b3bb75'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:10.037904 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9d2325f6-f776-4e20-9358-b43c2e825bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '493b1466e40a41319e02bc6711b3bb75'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:10.038697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39433a-5c82-427d-b723-157255ff08e6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:10.038986 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9d2325f6-f776-4e20-9358-b43c2e825bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '493b1466e40a41319e02bc6711b3bb75'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:10.048271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9d2325f6-f776-4e20-9358-b43c2e825bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '493b1466e40a41319e02bc6711b3bb75'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:10.055771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 939/1852] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:10.060656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39433a-5c82-427d-b723-157255ff08e6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:10.061043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9d2325f6-f776-4e20-9358-b43c2e825bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '493b1466e40a41319e02bc6711b3bb75'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:10.062734 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bf39433a-5c82-427d-b723-157255ff08e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:10.081625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39433a-5c82-427d-b723-157255ff08e6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:10.087688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329983d-ab43-422e-b3e5-7f43a9a24dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:10.088862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329983d-ab43-422e-b3e5-7f43a9a24dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:10.089150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329983d-ab43-422e-b3e5-7f43a9a24dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:10.089414 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329983d-ab43-422e-b3e5-7f43a9a24dc3 admin admin] PATH_INFO: `/v3/projects/182664d0bdfd4a25bdefd0912fa2ea03/users/5dff8925e4984c42933a0125a940121b/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:10.101230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 914/1853] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:09 2022] DELETE /identity/v3/users/493b1466e40a41319e02bc6711b3bb75 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:10.102724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24177910-0310-4c45-ab74-e32adb699b3d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:10.103020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24177910-0310-4c45-ab74-e32adb699b3d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:10.103302 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24177910-0310-4c45-ab74-e32adb699b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:10.108237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329983d-ab43-422e-b3e5-7f43a9a24dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=182664d0bdfd4a25bdefd0912fa2ea03, user_id=5dff8925e4984c42933a0125a940121b, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:10.111909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329983d-ab43-422e-b3e5-7f43a9a24dc3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:10.132357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-24177910-0310-4c45-ab74-e32adb699b3d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:10.132357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-24177910-0310-4c45-ab74-e32adb699b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:10.139077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 940/1854] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:10 2022] PUT /identity/v3/projects/182664d0bdfd4a25bdefd0912fa2ea03/users/5dff8925e4984c42933a0125a940121b/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:10.150603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df6d95-1269-4f23-8d98-eb4730344543 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:10.150969 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df6d95-1269-4f23-8d98-eb4730344543 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:10.151312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df6d95-1269-4f23-8d98-eb4730344543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:10.191297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-75df6d95-1269-4f23-8d98-eb4730344543 None None] MFA Rules not processed for user `5dff8925e4984c42933a0125a940121b`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:10.192502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-75df6d95-1269-4f23-8d98-eb4730344543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:10.213275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 915/1855] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:10.225312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24177910-0310-4c45-ab74-e32adb699b3d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:10.226756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-24177910-0310-4c45-ab74-e32adb699b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:10.284344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:10.292312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24177910-0310-4c45-ab74-e32adb699b3d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:10.293128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 941/1856] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:10.294289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df6d95-1269-4f23-8d98-eb4730344543 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:10.294409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e8072-1271-45ae-92a3-03793e238164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:10.294694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e8072-1271-45ae-92a3-03793e238164 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:10.295017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e8072-1271-45ae-92a3-03793e238164 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:10.295282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e8072-1271-45ae-92a3-03793e238164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:10.304977 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e8072-1271-45ae-92a3-03793e238164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:10.306481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e8072-1271-45ae-92a3-03793e238164 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:10.307089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df6d95-1269-4f23-8d98-eb4730344543 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:10.313954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5523e-634f-4d0a-8d54-fc70f3440a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:10.314654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5523e-634f-4d0a-8d54-fc70f3440a73 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:10.314889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5523e-634f-4d0a-8d54-fc70f3440a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:10.315150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5523e-634f-4d0a-8d54-fc70f3440a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:10.315962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5523e-634f-4d0a-8d54-fc70f3440a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:10.318019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5523e-634f-4d0a-8d54-fc70f3440a73 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:10.318593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 916/1857] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:10.359326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 942/1858] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:10 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:10.364075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e8072-1271-45ae-92a3-03793e238164 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:10.377701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e8072-1271-45ae-92a3-03793e238164 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:10.517780 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185b822-a3bf-420a-8168-5323cc29bad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:10.519608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185b822-a3bf-420a-8168-5323cc29bad9 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:10.519927 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185b822-a3bf-420a-8168-5323cc29bad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:10.522219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185b822-a3bf-420a-8168-5323cc29bad9 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:10.523562 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185b822-a3bf-420a-8168-5323cc29bad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:10.525810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185b822-a3bf-420a-8168-5323cc29bad9 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:10.546714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:10.573171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 917/1859] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:10 2022] POST /identity/v3/users => generated 299 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:10.578268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5523e-634f-4d0a-8d54-fc70f3440a73 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:10.589044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5523e-634f-4d0a-8d54-fc70f3440a73 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:10.594214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b5ad7-a2bd-4923-8034-2a3829512dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:10.595632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b5ad7-a2bd-4923-8034-2a3829512dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:10.595632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b5ad7-a2bd-4923-8034-2a3829512dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:10.595632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b5ad7-a2bd-4923-8034-2a3829512dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:10.596198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b5ad7-a2bd-4923-8034-2a3829512dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:10.597978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b5ad7-a2bd-4923-8034-2a3829512dd8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:10.609962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 943/1860] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:10.614099 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185b822-a3bf-420a-8168-5323cc29bad9 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:10.628454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185b822-a3bf-420a-8168-5323cc29bad9 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:10.637732 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2641f-3caa-40c1-bb37-abb0cbfb617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:10.637732 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2641f-3caa-40c1-bb37-abb0cbfb617d admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:10.637732 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2641f-3caa-40c1-bb37-abb0cbfb617d admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:10.638178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2641f-3caa-40c1-bb37-abb0cbfb617d admin admin] PATH_INFO: `/v3/projects/b86ffc5fe542422a981364ae05b574bc/users/93f173fa5c1948039fd262e0934faa9a/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:10.660427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2641f-3caa-40c1-bb37-abb0cbfb617d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b86ffc5fe542422a981364ae05b574bc, user_id=93f173fa5c1948039fd262e0934faa9a, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:10.664110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2641f-3caa-40c1-bb37-abb0cbfb617d admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:10.694377 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 918/1861] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:10 2022] PUT /identity/v3/projects/b86ffc5fe542422a981364ae05b574bc/users/93f173fa5c1948039fd262e0934faa9a/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:10.696248 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b5ad7-a2bd-4923-8034-2a3829512dd8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:10.711070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b5ad7-a2bd-4923-8034-2a3829512dd8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:10.819612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798416fd-4ea2-466e-bb5f-399ff233b06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:10.820365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798416fd-4ea2-466e-bb5f-399ff233b06e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:10.820720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798416fd-4ea2-466e-bb5f-399ff233b06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:10.821065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798416fd-4ea2-466e-bb5f-399ff233b06e admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:10.822223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798416fd-4ea2-466e-bb5f-399ff233b06e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:10.826159 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798416fd-4ea2-466e-bb5f-399ff233b06e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:10.836713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 944/1862] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:10.840859 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2641f-3caa-40c1-bb37-abb0cbfb617d admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:10.855998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2641f-3caa-40c1-bb37-abb0cbfb617d admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:10.864451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0722c3-f361-4098-b71a-59cf01460e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:10.864999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0722c3-f361-4098-b71a-59cf01460e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:10.865297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0722c3-f361-4098-b71a-59cf01460e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:10.865688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0722c3-f361-4098-b71a-59cf01460e9d admin admin] PATH_INFO: `/v3/projects/b86ffc5fe542422a981364ae05b574bc/users/93f173fa5c1948039fd262e0934faa9a/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:10.868752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0722c3-f361-4098-b71a-59cf01460e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b86ffc5fe542422a981364ae05b574bc, user_id=93f173fa5c1948039fd262e0934faa9a, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:10.871955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0722c3-f361-4098-b71a-59cf01460e9d admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:10.897251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 919/1863] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:10 2022] PUT /identity/v3/projects/b86ffc5fe542422a981364ae05b574bc/users/93f173fa5c1948039fd262e0934faa9a/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:10.910070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c4c4e-1ba6-4dfe-b04a-ba64dbf232bc None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:10.910070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c4c4e-1ba6-4dfe-b04a-ba64dbf232bc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:10.910070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c4c4e-1ba6-4dfe-b04a-ba64dbf232bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:10.946702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-cd2c4c4e-1ba6-4dfe-b04a-ba64dbf232bc None None] MFA Rules not processed for user `93f173fa5c1948039fd262e0934faa9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:10.947927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cd2c4c4e-1ba6-4dfe-b04a-ba64dbf232bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:10.950901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0722c3-f361-4098-b71a-59cf01460e9d admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:10.953060 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ad0722c3-f361-4098-b71a-59cf01460e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:11.058572 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0722c3-f361-4098-b71a-59cf01460e9d admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:11.065647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:11.066543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:11.066787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:11.067341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] PATH_INFO: `/v3/projects/9e0c8d7c32674536907ba51031b5e05e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:11.071576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0c8d7c32674536907ba51031b5e05e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:11.073965 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:11.129886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 945/1864] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:11.144364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c4c4e-1ba6-4dfe-b04a-ba64dbf232bc None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:11.145635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cd2c4c4e-1ba6-4dfe-b04a-ba64dbf232bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:11.207438 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0c8d7c32674536907ba51031b5e05e', 'request_id': 'req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.220762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0c8d7c32674536907ba51031b5e05e', 'request_id': 'req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.235677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0c8d7c32674536907ba51031b5e05e', 'request_id': 'req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.251275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:11.261205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c4c4e-1ba6-4dfe-b04a-ba64dbf232bc None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:11.262645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] The token cache is being invalidate because project 9e0c8d7c32674536907ba51031b5e05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:11.262948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.264338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.268402 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 920/1865] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:10 2022] DELETE /identity/v3/projects/9e0c8d7c32674536907ba51031b5e05e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:11.272497 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:11.275546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:11.356041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644240a-e767-4973-b950-e7f2b8bba37c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:11.357262 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644240a-e767-4973-b950-e7f2b8bba37c service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:11.358279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644240a-e767-4973-b950-e7f2b8bba37c service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:11.358716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644240a-e767-4973-b950-e7f2b8bba37c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:11.361566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8644240a-e767-4973-b950-e7f2b8bba37c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:11.370928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:11.405509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166feb0-9976-41ef-a98d-3c5bd1f6dec8 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:11.420048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458c45f-df27-4b95-9f1a-bf93c57dbd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:11.421236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458c45f-df27-4b95-9f1a-bf93c57dbd2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:11.421734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458c45f-df27-4b95-9f1a-bf93c57dbd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:11.422274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458c45f-df27-4b95-9f1a-bf93c57dbd2e admin admin] PATH_INFO: `/v3/users/6b4d67ffa91a4cad977e86e032abc6ec` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:11.423792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458c45f-df27-4b95-9f1a-bf93c57dbd2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b4d67ffa91a4cad977e86e032abc6ec)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:11.425853 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458c45f-df27-4b95-9f1a-bf93c57dbd2e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:11.471920 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644240a-e767-4973-b950-e7f2b8bba37c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:11.474727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644240a-e767-4973-b950-e7f2b8bba37c service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:11.488954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 946/1866] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:40:11.492508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-4458c45f-df27-4b95-9f1a-bf93c57dbd2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b4d67ffa91a4cad977e86e032abc6ec'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.493251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-4458c45f-df27-4b95-9f1a-bf93c57dbd2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b4d67ffa91a4cad977e86e032abc6ec'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.494225 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-4458c45f-df27-4b95-9f1a-bf93c57dbd2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b4d67ffa91a4cad977e86e032abc6ec'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.512669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-4458c45f-df27-4b95-9f1a-bf93c57dbd2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b4d67ffa91a4cad977e86e032abc6ec'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.528656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-4458c45f-df27-4b95-9f1a-bf93c57dbd2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b4d67ffa91a4cad977e86e032abc6ec'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.564852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 921/1867] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:11 2022] DELETE /identity/v3/users/6b4d67ffa91a4cad977e86e032abc6ec => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:11.635923 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644240a-e767-4973-b950-e7f2b8bba37c service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:11.638167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8644240a-e767-4973-b950-e7f2b8bba37c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:11.658359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644240a-e767-4973-b950-e7f2b8bba37c service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:11.737315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:11.738253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:11.738539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:11.738822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] PATH_INFO: `/v3/users/5dff8925e4984c42933a0125a940121b` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:11.741385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dff8925e4984c42933a0125a940121b)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:11.743595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:11.809348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dff8925e4984c42933a0125a940121b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.824793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dff8925e4984c42933a0125a940121b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.828064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dff8925e4984c42933a0125a940121b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.831232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:11.840445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dff8925e4984c42933a0125a940121b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.843489 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dff8925e4984c42933a0125a940121b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:11.880567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb951a-207c-4efb-ae78-31896daf5052 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:11.880856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb951a-207c-4efb-ae78-31896daf5052 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:11.881126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb951a-207c-4efb-ae78-31896daf5052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:11.884505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 947/1868] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:11 2022] DELETE /identity/v3/users/5dff8925e4984c42933a0125a940121b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:11.906038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-79fb951a-207c-4efb-ae78-31896daf5052 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:11.907201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-79fb951a-207c-4efb-ae78-31896daf5052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:12.017888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:12.020294 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:12.029327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 922/1869] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:12.050174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb951a-207c-4efb-ae78-31896daf5052 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:12.050174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-79fb951a-207c-4efb-ae78-31896daf5052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:12.055367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1d9aa-23f7-4bad-8c04-2562cde38ed2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:12.062137 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600c845-864b-43f2-b748-a392fa4c4cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:12.063469 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600c845-864b-43f2-b748-a392fa4c4cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.063909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600c845-864b-43f2-b748-a392fa4c4cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.064322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600c845-864b-43f2-b748-a392fa4c4cc4 admin admin] PATH_INFO: `/v3/projects/7f0d8bbf9e704c1ba97b69c6529f43f7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.065834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600c845-864b-43f2-b748-a392fa4c4cc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f0d8bbf9e704c1ba97b69c6529f43f7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:12.068031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600c845-864b-43f2-b748-a392fa4c4cc4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:12.131187 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb951a-207c-4efb-ae78-31896daf5052 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:12.135804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5600c845-864b-43f2-b748-a392fa4c4cc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f0d8bbf9e704c1ba97b69c6529f43f7', 'request_id': 'req-5600c845-864b-43f2-b748-a392fa4c4cc4'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.148379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5600c845-864b-43f2-b748-a392fa4c4cc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f0d8bbf9e704c1ba97b69c6529f43f7', 'request_id': 'req-5600c845-864b-43f2-b748-a392fa4c4cc4'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.154860 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:12.155472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5600c845-864b-43f2-b748-a392fa4c4cc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f0d8bbf9e704c1ba97b69c6529f43f7', 'request_id': 'req-5600c845-864b-43f2-b748-a392fa4c4cc4'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.176433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5600c845-864b-43f2-b748-a392fa4c4cc4 admin admin] The token cache is being invalidate because project 7f0d8bbf9e704c1ba97b69c6529f43f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:12.176867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5600c845-864b-43f2-b748-a392fa4c4cc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.177584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5600c845-864b-43f2-b748-a392fa4c4cc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.182406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 948/1870] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:12 2022] DELETE /identity/v3/projects/7f0d8bbf9e704c1ba97b69c6529f43f7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:12.189277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a03c8-4f44-4c65-aa38-e8342c7b71e6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.189575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a03c8-4f44-4c65-aa38-e8342c7b71e6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.189858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a03c8-4f44-4c65-aa38-e8342c7b71e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.195417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be631a3a-abc0-44be-8fbf-1e33aa5b3a55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:12.196524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be631a3a-abc0-44be-8fbf-1e33aa5b3a55 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.197154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be631a3a-abc0-44be-8fbf-1e33aa5b3a55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.197649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be631a3a-abc0-44be-8fbf-1e33aa5b3a55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.199618 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-be631a3a-abc0-44be-8fbf-1e33aa5b3a55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:12.211602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-fc1a03c8-4f44-4c65-aa38-e8342c7b71e6 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:12.212723 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fc1a03c8-4f44-4c65-aa38-e8342c7b71e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:12.274820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be631a3a-abc0-44be-8fbf-1e33aa5b3a55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:12.276377 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be631a3a-abc0-44be-8fbf-1e33aa5b3a55 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:12.286079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 923/1871] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:40:12.286486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 949/1872] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:12.294012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7a7e0-7d27-4a98-8bac-4c6ae0fe1be2 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.294328 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7a7e0-7d27-4a98-8bac-4c6ae0fe1be2 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.294610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7a7e0-7d27-4a98-8bac-4c6ae0fe1be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.315724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b0c7a7e0-7d27-4a98-8bac-4c6ae0fe1be2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:12.316928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b0c7a7e0-7d27-4a98-8bac-4c6ae0fe1be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:12.325162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 924/1873] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:12.329350 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a03c8-4f44-4c65-aa38-e8342c7b71e6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:12.344412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a03c8-4f44-4c65-aa38-e8342c7b71e6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:12.363288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041d411-f962-4e8c-b857-a18a3888d13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:12.364237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041d411-f962-4e8c-b857-a18a3888d13f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.364557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041d411-f962-4e8c-b857-a18a3888d13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.364902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041d411-f962-4e8c-b857-a18a3888d13f admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.365802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041d411-f962-4e8c-b857-a18a3888d13f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:12.366149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041d411-f962-4e8c-b857-a18a3888d13f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:12.367772 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041d411-f962-4e8c-b857-a18a3888d13f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:12.381867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 950/1874] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:40:12.386088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7a7e0-7d27-4a98-8bac-4c6ae0fe1be2 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:12.397584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7a7e0-7d27-4a98-8bac-4c6ae0fe1be2 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:12.402861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770aafa3-45d2-4ac7-a18b-bc8caa6d634e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:12.403649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770aafa3-45d2-4ac7-a18b-bc8caa6d634e admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.404085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770aafa3-45d2-4ac7-a18b-bc8caa6d634e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.404532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770aafa3-45d2-4ac7-a18b-bc8caa6d634e admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.406027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770aafa3-45d2-4ac7-a18b-bc8caa6d634e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:12.407583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770aafa3-45d2-4ac7-a18b-bc8caa6d634e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:12.445663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 925/1875] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:12 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:12.446831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041d411-f962-4e8c-b857-a18a3888d13f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:12.470595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041d411-f962-4e8c-b857-a18a3888d13f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:12.540900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:12.577480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d4664-28e3-4154-8cd5-08f82080cb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:12.577480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d4664-28e3-4154-8cd5-08f82080cb26 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.577480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d4664-28e3-4154-8cd5-08f82080cb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.577480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d4664-28e3-4154-8cd5-08f82080cb26 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.577950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d4664-28e3-4154-8cd5-08f82080cb26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:12.577950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d4664-28e3-4154-8cd5-08f82080cb26 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:12.602198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770aafa3-45d2-4ac7-a18b-bc8caa6d634e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:12.602851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 951/1876] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:12 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:12.614500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770aafa3-45d2-4ac7-a18b-bc8caa6d634e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:12.624262 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d4664-28e3-4154-8cd5-08f82080cb26 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:12.626121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d39d4664-28e3-4154-8cd5-08f82080cb26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:12.629742 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1150ed-6687-4988-be1f-e49aa326e55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:12.630459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1150ed-6687-4988-be1f-e49aa326e55d admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.630743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1150ed-6687-4988-be1f-e49aa326e55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.631017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1150ed-6687-4988-be1f-e49aa326e55d admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.643700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1150ed-6687-4988-be1f-e49aa326e55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:12.643885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1150ed-6687-4988-be1f-e49aa326e55d admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:12.654624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d4664-28e3-4154-8cd5-08f82080cb26 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:12.665528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 926/1877] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:12.666299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:12.666451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1150ed-6687-4988-be1f-e49aa326e55d admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:12.667327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.667684 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.671940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] PATH_INFO: `/v3/projects/182664d0bdfd4a25bdefd0912fa2ea03` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.676541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1150ed-6687-4988-be1f-e49aa326e55d admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:12.682181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e13fad-e0fe-43e6-bc66-52ab5622115c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:12.682869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=182664d0bdfd4a25bdefd0912fa2ea03)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:12.683512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e13fad-e0fe-43e6-bc66-52ab5622115c admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.683908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e13fad-e0fe-43e6-bc66-52ab5622115c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.684380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e13fad-e0fe-43e6-bc66-52ab5622115c admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.686023 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e13fad-e0fe-43e6-bc66-52ab5622115c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:12.686821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:12.688154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e13fad-e0fe-43e6-bc66-52ab5622115c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:12.700519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 927/1878] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:12.704787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e13fad-e0fe-43e6-bc66-52ab5622115c admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:12.714762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e13fad-e0fe-43e6-bc66-52ab5622115c admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:12.720414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087fa47-c0ea-4b39-9461-5a203c2231e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:12.721609 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087fa47-c0ea-4b39-9461-5a203c2231e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.722103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087fa47-c0ea-4b39-9461-5a203c2231e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.722422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087fa47-c0ea-4b39-9461-5a203c2231e7 admin admin] PATH_INFO: `/v3/projects/88f7790fae3846db911734a468a664ab/users/a20085e2d5e945dcbc8cc1c121f3bde2/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.740369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087fa47-c0ea-4b39-9461-5a203c2231e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f7790fae3846db911734a468a664ab, user_id=a20085e2d5e945dcbc8cc1c121f3bde2, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:12.743075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087fa47-c0ea-4b39-9461-5a203c2231e7 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:12.750433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182664d0bdfd4a25bdefd0912fa2ea03', 'request_id': 'req-d941174c-91b9-4622-8321-bb5b92503a07'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.757276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 928/1879] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:12 2022] PUT /identity/v3/projects/88f7790fae3846db911734a468a664ab/users/a20085e2d5e945dcbc8cc1c121f3bde2/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:12.759115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182664d0bdfd4a25bdefd0912fa2ea03', 'request_id': 'req-d941174c-91b9-4622-8321-bb5b92503a07'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.766413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182664d0bdfd4a25bdefd0912fa2ea03', 'request_id': 'req-d941174c-91b9-4622-8321-bb5b92503a07'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.768214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628944d-8d9e-4175-925f-7c3c2de1de02 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.768581 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628944d-8d9e-4175-925f-7c3c2de1de02 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.768821 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628944d-8d9e-4175-925f-7c3c2de1de02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.782849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] The token cache is being invalidate because project 182664d0bdfd4a25bdefd0912fa2ea03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:12.782849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.783567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.785207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 952/1880] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:12 2022] DELETE /identity/v3/projects/182664d0bdfd4a25bdefd0912fa2ea03 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:12.789330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:12.790806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:12.810121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b628944d-8d9e-4175-925f-7c3c2de1de02 None None] MFA Rules not processed for user `a20085e2d5e945dcbc8cc1c121f3bde2`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:12.811954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b628944d-8d9e-4175-925f-7c3c2de1de02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:12.855894 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:12.857548 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941174c-91b9-4622-8321-bb5b92503a07 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:12.879176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 929/1881] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:12.883550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628944d-8d9e-4175-925f-7c3c2de1de02 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:12.885260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b628944d-8d9e-4175-925f-7c3c2de1de02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:12.916938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:12.917842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.918131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.918408 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] PATH_INFO: `/v3/users/93f173fa5c1948039fd262e0934faa9a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.919617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93f173fa5c1948039fd262e0934faa9a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:12.922218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:12.927354 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:12.961535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628944d-8d9e-4175-925f-7c3c2de1de02 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:12.966360 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dcde6-5b02-44be-bcf4-11663692a710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:12.967051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dcde6-5b02-44be-bcf4-11663692a710 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:12.967335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dcde6-5b02-44be-bcf4-11663692a710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:12.967612 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dcde6-5b02-44be-bcf4-11663692a710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:12.968319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dcde6-5b02-44be-bcf4-11663692a710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:12.970309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dcde6-5b02-44be-bcf4-11663692a710 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:12.983131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93f173fa5c1948039fd262e0934faa9a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.992465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93f173fa5c1948039fd262e0934faa9a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.993199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93f173fa5c1948039fd262e0934faa9a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:12.999893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 930/1882] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:12 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:13.002833 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93f173fa5c1948039fd262e0934faa9a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:13.003499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93f173fa5c1948039fd262e0934faa9a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:13.003752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dcde6-5b02-44be-bcf4-11663692a710 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.005397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-647dcde6-5b02-44be-bcf4-11663692a710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:13.026747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 953/1883] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:12 2022] DELETE /identity/v3/users/93f173fa5c1948039fd262e0934faa9a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:13.076279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dcde6-5b02-44be-bcf4-11663692a710 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.157968 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab369c-4967-4e9a-b231-b79cb755e840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.158746 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab369c-4967-4e9a-b231-b79cb755e840 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.159106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab369c-4967-4e9a-b231-b79cb755e840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.159818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab369c-4967-4e9a-b231-b79cb755e840 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.160265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab369c-4967-4e9a-b231-b79cb755e840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.162844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab369c-4967-4e9a-b231-b79cb755e840 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.210603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 931/1884] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:13 2022] POST /identity/v3/users => generated 298 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:13.214431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.225216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9723b64-70b1-4ce7-964d-695a43b317f4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.230136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff16dc-02fd-4a0b-80ae-e94d2e1cb2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.230802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff16dc-02fd-4a0b-80ae-e94d2e1cb2ba admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.231081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff16dc-02fd-4a0b-80ae-e94d2e1cb2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.231365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff16dc-02fd-4a0b-80ae-e94d2e1cb2ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.232099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff16dc-02fd-4a0b-80ae-e94d2e1cb2ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.237479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff16dc-02fd-4a0b-80ae-e94d2e1cb2ba admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.251194 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 954/1885] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:13.255387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab369c-4967-4e9a-b231-b79cb755e840 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.267138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab369c-4967-4e9a-b231-b79cb755e840 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.273286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6b020-115b-4f62-adef-553c78d7b142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.274418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6b020-115b-4f62-adef-553c78d7b142 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.274795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6b020-115b-4f62-adef-553c78d7b142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.275466 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6b020-115b-4f62-adef-553c78d7b142 admin admin] PATH_INFO: `/v3/projects/4adc50f6a89f4792a617c8ce265219f4/users/4658ee5cc14e418a91e7dd4226f23693/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.277962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:13.295933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6b020-115b-4f62-adef-553c78d7b142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4adc50f6a89f4792a617c8ce265219f4, user_id=4658ee5cc14e418a91e7dd4226f23693, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.298841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6b020-115b-4f62-adef-553c78d7b142 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.319307 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 932/1886] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:13 2022] PUT /identity/v3/projects/4adc50f6a89f4792a617c8ce265219f4/users/4658ee5cc14e418a91e7dd4226f23693/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:13.320491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff16dc-02fd-4a0b-80ae-e94d2e1cb2ba admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.328408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6b020-115b-4f62-adef-553c78d7b142 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.331477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-48e6b020-115b-4f62-adef-553c78d7b142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:13.332721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff16dc-02fd-4a0b-80ae-e94d2e1cb2ba admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.405813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a03a63-4fd1-4d4b-afb8-5eea4f5ebdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.406706 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a03a63-4fd1-4d4b-afb8-5eea4f5ebdae admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.406859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a03a63-4fd1-4d4b-afb8-5eea4f5ebdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.407118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a03a63-4fd1-4d4b-afb8-5eea4f5ebdae admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.407855 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a03a63-4fd1-4d4b-afb8-5eea4f5ebdae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.409467 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a03a63-4fd1-4d4b-afb8-5eea4f5ebdae admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.411993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6b020-115b-4f62-adef-553c78d7b142 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.416643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9ebcd-c614-49ac-bc30-49d929a0f7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.417506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9ebcd-c614-49ac-bc30-49d929a0f7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.417788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9ebcd-c614-49ac-bc30-49d929a0f7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.418128 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9ebcd-c614-49ac-bc30-49d929a0f7be admin admin] PATH_INFO: `/v3/projects/b86ffc5fe542422a981364ae05b574bc` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.419384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9ebcd-c614-49ac-bc30-49d929a0f7be admin admin] RBAC: Authorizing `identity:delete_project(project_id=b86ffc5fe542422a981364ae05b574bc)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.421295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 955/1887] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:13.424920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9ebcd-c614-49ac-bc30-49d929a0f7be admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.425168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a03a63-4fd1-4d4b-afb8-5eea4f5ebdae admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.437637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a03a63-4fd1-4d4b-afb8-5eea4f5ebdae admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.443298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61cca1-88b2-474d-9ffc-7d007da2b66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.444318 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61cca1-88b2-474d-9ffc-7d007da2b66c admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.444841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61cca1-88b2-474d-9ffc-7d007da2b66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.445214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61cca1-88b2-474d-9ffc-7d007da2b66c admin admin] PATH_INFO: `/v3/projects/4adc50f6a89f4792a617c8ce265219f4/users/4658ee5cc14e418a91e7dd4226f23693/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.447589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61cca1-88b2-474d-9ffc-7d007da2b66c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4adc50f6a89f4792a617c8ce265219f4, user_id=4658ee5cc14e418a91e7dd4226f23693, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.450626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61cca1-88b2-474d-9ffc-7d007da2b66c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.464539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 956/1888] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:13 2022] PUT /identity/v3/projects/4adc50f6a89f4792a617c8ce265219f4/users/4658ee5cc14e418a91e7dd4226f23693/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:13.476310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5955c6d-6164-492a-80b7-bcb464225d47 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.476611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5955c6d-6164-492a-80b7-bcb464225d47 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.476894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5955c6d-6164-492a-80b7-bcb464225d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.479324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-71b9ebcd-c614-49ac-bc30-49d929a0f7be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b86ffc5fe542422a981364ae05b574bc', 'request_id': 'req-71b9ebcd-c614-49ac-bc30-49d929a0f7be'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:13.486489 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-71b9ebcd-c614-49ac-bc30-49d929a0f7be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b86ffc5fe542422a981364ae05b574bc', 'request_id': 'req-71b9ebcd-c614-49ac-bc30-49d929a0f7be'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:13.491948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-71b9ebcd-c614-49ac-bc30-49d929a0f7be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b86ffc5fe542422a981364ae05b574bc', 'request_id': 'req-71b9ebcd-c614-49ac-bc30-49d929a0f7be'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:13.505144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d5955c6d-6164-492a-80b7-bcb464225d47 None None] MFA Rules not processed for user `4658ee5cc14e418a91e7dd4226f23693`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:13.505472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-71b9ebcd-c614-49ac-bc30-49d929a0f7be admin admin] The token cache is being invalidate because project b86ffc5fe542422a981364ae05b574bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:13.505750 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-71b9ebcd-c614-49ac-bc30-49d929a0f7be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:13.506374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d5955c6d-6164-492a-80b7-bcb464225d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:13.506481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-71b9ebcd-c614-49ac-bc30-49d929a0f7be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:13.507949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 933/1889] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:13 2022] DELETE /identity/v3/projects/b86ffc5fe542422a981364ae05b574bc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:13.513574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eeac7d-3535-4660-948c-d9d356b86309 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.514004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eeac7d-3535-4660-948c-d9d356b86309 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.514831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eeac7d-3535-4660-948c-d9d356b86309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.534641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-69eeac7d-3535-4660-948c-d9d356b86309 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:13.535846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-69eeac7d-3535-4660-948c-d9d356b86309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:13.551659 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:13.578948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 957/1890] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:13.591379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5955c6d-6164-492a-80b7-bcb464225d47 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.594249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d5955c6d-6164-492a-80b7-bcb464225d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:13.597612 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 934/1891] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:13.605381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8702f-2e64-4031-99e7-df0112d8b0ab None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.606010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8702f-2e64-4031-99e7-df0112d8b0ab None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.606010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8702f-2e64-4031-99e7-df0112d8b0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.627063 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-c0b8702f-2e64-4031-99e7-df0112d8b0ab None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:13.628167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c0b8702f-2e64-4031-99e7-df0112d8b0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:13.636039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 935/1892] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:13.640658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8702f-2e64-4031-99e7-df0112d8b0ab None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.657207 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8702f-2e64-4031-99e7-df0112d8b0ab None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.668077 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cf8ad-7268-4bf5-8f5d-da3dddf8d293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.668714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5955c6d-6164-492a-80b7-bcb464225d47 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.668864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cf8ad-7268-4bf5-8f5d-da3dddf8d293 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.669167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cf8ad-7268-4bf5-8f5d-da3dddf8d293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.669449 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cf8ad-7268-4bf5-8f5d-da3dddf8d293 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.670290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643cf8ad-7268-4bf5-8f5d-da3dddf8d293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:13.670604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643cf8ad-7268-4bf5-8f5d-da3dddf8d293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.672213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643cf8ad-7268-4bf5-8f5d-da3dddf8d293 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.672489 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda03605-16a0-4c3d-943f-fb783429ab72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.673140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda03605-16a0-4c3d-943f-fb783429ab72 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.673421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda03605-16a0-4c3d-943f-fb783429ab72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.673696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda03605-16a0-4c3d-943f-fb783429ab72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.674127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:13.685001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda03605-16a0-4c3d-943f-fb783429ab72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.686536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda03605-16a0-4c3d-943f-fb783429ab72 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.688676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 936/1893] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:40:13.693258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cf8ad-7268-4bf5-8f5d-da3dddf8d293 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.698975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 958/1894] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:40:13.705297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cf8ad-7268-4bf5-8f5d-da3dddf8d293 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.710829 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a6882-2069-4016-89f8-23ec94dd0899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.711668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a6882-2069-4016-89f8-23ec94dd0899 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.711961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a6882-2069-4016-89f8-23ec94dd0899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.712235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a6882-2069-4016-89f8-23ec94dd0899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.713411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a6882-2069-4016-89f8-23ec94dd0899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.718521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a6882-2069-4016-89f8-23ec94dd0899 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.748012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 937/1895] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:13 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:13.752740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda03605-16a0-4c3d-943f-fb783429ab72 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.764653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda03605-16a0-4c3d-943f-fb783429ab72 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.808796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a6882-2069-4016-89f8-23ec94dd0899 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.851140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e9a6b-6b88-4f3c-8835-be712e96b86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.851946 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e9a6b-6b88-4f3c-8835-be712e96b86b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.852235 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e9a6b-6b88-4f3c-8835-be712e96b86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.852514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e9a6b-6b88-4f3c-8835-be712e96b86b admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.853732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e9a6b-6b88-4f3c-8835-be712e96b86b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.856183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e9a6b-6b88-4f3c-8835-be712e96b86b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.870487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a6882-2069-4016-89f8-23ec94dd0899 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.875256 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d6ede-6842-4ef3-919a-0c743c1ca8f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.875967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d6ede-6842-4ef3-919a-0c743c1ca8f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.876287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d6ede-6842-4ef3-919a-0c743c1ca8f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.876567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d6ede-6842-4ef3-919a-0c743c1ca8f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.878942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b96d6ede-6842-4ef3-919a-0c743c1ca8f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:13.887942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 959/1896] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:13 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:13.889861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e9a6b-6b88-4f3c-8835-be712e96b86b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.899869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e9a6b-6b88-4f3c-8835-be712e96b86b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.905625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776c339-53ae-4e46-8304-2b757dee66d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.906332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776c339-53ae-4e46-8304-2b757dee66d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.906616 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776c339-53ae-4e46-8304-2b757dee66d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.906894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776c339-53ae-4e46-8304-2b757dee66d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.907664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776c339-53ae-4e46-8304-2b757dee66d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.909288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776c339-53ae-4e46-8304-2b757dee66d1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.919890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 960/1897] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:13.923609 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776c339-53ae-4e46-8304-2b757dee66d1 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.934486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776c339-53ae-4e46-8304-2b757dee66d1 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.939144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f587c-b0b4-49f2-87a1-26287db55589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.939814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f587c-b0b4-49f2-87a1-26287db55589 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.940289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f587c-b0b4-49f2-87a1-26287db55589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.940583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f587c-b0b4-49f2-87a1-26287db55589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:13.941932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f587c-b0b4-49f2-87a1-26287db55589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.943555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f587c-b0b4-49f2-87a1-26287db55589 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.948515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d6ede-6842-4ef3-919a-0c743c1ca8f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:13.950088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d6ede-6842-4ef3-919a-0c743c1ca8f4 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:13.957617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 961/1898] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:13.961165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 938/1899] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:40:13.961643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d6ede-6842-4ef3-919a-0c743c1ca8f4 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:13.973848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d6ede-6842-4ef3-919a-0c743c1ca8f4 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:13.978467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee0a11-ff8d-490f-a3d8-1ba46bfb129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:13.982170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee0a11-ff8d-490f-a3d8-1ba46bfb129d admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:13.982170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee0a11-ff8d-490f-a3d8-1ba46bfb129d admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:13.982170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee0a11-ff8d-490f-a3d8-1ba46bfb129d admin admin] PATH_INFO: `/v3/projects/05f49dcc78354ab99644c6fb44dce72c/users/4c22642d48464b1795552308a220e437/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:14.001851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee0a11-ff8d-490f-a3d8-1ba46bfb129d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f49dcc78354ab99644c6fb44dce72c, user_id=4c22642d48464b1795552308a220e437, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:14.004646 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee0a11-ff8d-490f-a3d8-1ba46bfb129d admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:14.020063 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 939/1900] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:13 2022] PUT /identity/v3/projects/05f49dcc78354ab99644c6fb44dce72c/users/4c22642d48464b1795552308a220e437/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:14.031472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185be842-e6c6-492a-8693-6945c7baa9ef None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:14.031834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185be842-e6c6-492a-8693-6945c7baa9ef None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:14.032063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185be842-e6c6-492a-8693-6945c7baa9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:14.059709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-185be842-e6c6-492a-8693-6945c7baa9ef None None] MFA Rules not processed for user `4c22642d48464b1795552308a220e437`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:14.061487 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-185be842-e6c6-492a-8693-6945c7baa9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:14.101456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:14.133955 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 962/1901] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:14.208276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee0a11-ff8d-490f-a3d8-1ba46bfb129d admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:14.209976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0bee0a11-ff8d-490f-a3d8-1ba46bfb129d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:14.322391 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee0a11-ff8d-490f-a3d8-1ba46bfb129d admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:14.330265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20f9bc-53e1-4460-a63a-d1b8fc2e6ff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:14.331149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20f9bc-53e1-4460-a63a-d1b8fc2e6ff7 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:14.331499 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20f9bc-53e1-4460-a63a-d1b8fc2e6ff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:14.331777 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20f9bc-53e1-4460-a63a-d1b8fc2e6ff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:14.349530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20f9bc-53e1-4460-a63a-d1b8fc2e6ff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:14.350988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20f9bc-53e1-4460-a63a-d1b8fc2e6ff7 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:14.360654 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 940/1902] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:14.382197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185be842-e6c6-492a-8693-6945c7baa9ef None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:14.444978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185be842-e6c6-492a-8693-6945c7baa9ef None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:14.449506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:14.450415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:14.450700 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:14.450974 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] PATH_INFO: `/v3/users/4c22642d48464b1795552308a220e437` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:14.452215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c22642d48464b1795552308a220e437)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:14.454401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:14.503882 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c22642d48464b1795552308a220e437'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:14.511550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c22642d48464b1795552308a220e437'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:14.512258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c22642d48464b1795552308a220e437'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:14.519775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c22642d48464b1795552308a220e437'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:14.520618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c22642d48464b1795552308a220e437'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:14.538901 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 963/1903] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:14 2022] DELETE /identity/v3/users/4c22642d48464b1795552308a220e437 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:14.543731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef34f1-6e08-46da-8aec-6c8270bdf090 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:14.544018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef34f1-6e08-46da-8aec-6c8270bdf090 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:14.544319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef34f1-6e08-46da-8aec-6c8270bdf090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:14.553086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:14.565818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-a5ef34f1-6e08-46da-8aec-6c8270bdf090 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:14.567109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a5ef34f1-6e08-46da-8aec-6c8270bdf090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:14.630929 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 941/1904] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:14.638799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:14.640673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:14.719528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c180d7f-c9f1-43ba-8b79-30ff758676af admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:14.724177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717e72f-6ba2-482e-9cf8-5f73e8223b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:14.725094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717e72f-6ba2-482e-9cf8-5f73e8223b99 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:14.725438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717e72f-6ba2-482e-9cf8-5f73e8223b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:14.725796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717e72f-6ba2-482e-9cf8-5f73e8223b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:14.736638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717e72f-6ba2-482e-9cf8-5f73e8223b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:14.738249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717e72f-6ba2-482e-9cf8-5f73e8223b99 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:14.765126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 964/1905] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:15.332038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef34f1-6e08-46da-8aec-6c8270bdf090 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:15.333863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a5ef34f1-6e08-46da-8aec-6c8270bdf090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:15.348297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef34f1-6e08-46da-8aec-6c8270bdf090 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:15.353693 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:15.354688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:15.355049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:15.355490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa admin admin] PATH_INFO: `/v3/projects/05f49dcc78354ab99644c6fb44dce72c` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:15.356695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f49dcc78354ab99644c6fb44dce72c)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:15.359352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:15.427433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f49dcc78354ab99644c6fb44dce72c', 'request_id': 'req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:15.435980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f49dcc78354ab99644c6fb44dce72c', 'request_id': 'req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:15.441765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f49dcc78354ab99644c6fb44dce72c', 'request_id': 'req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:15.454314 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa admin admin] The token cache is being invalidate because project 05f49dcc78354ab99644c6fb44dce72c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:15.454642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:15.455253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9ef98fb2-1f63-40f2-8442-e9025e39e1aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:15.456849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 942/1906] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:15 2022] DELETE /identity/v3/projects/05f49dcc78354ab99644c6fb44dce72c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:15.461867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2e143-039e-41d9-ba68-221708f584ab None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:15.462362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2e143-039e-41d9-ba68-221708f584ab None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:15.462463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2e143-039e-41d9-ba68-221708f584ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:15.470098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:15.488134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-8ed2e143-039e-41d9-ba68-221708f584ab None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:15.489775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8ed2e143-039e-41d9-ba68-221708f584ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:15.586507 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 965/1907] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:15.594745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdf6c5-f09a-4fe3-b21b-330471d824de None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:15.595348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdf6c5-f09a-4fe3-b21b-330471d824de None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:15.595730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdf6c5-f09a-4fe3-b21b-330471d824de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:15.620615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-75cdf6c5-f09a-4fe3-b21b-330471d824de None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:15.622290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-75cdf6c5-f09a-4fe3-b21b-330471d824de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:15.633872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 943/1908] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:15.638115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2e143-039e-41d9-ba68-221708f584ab None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:15.652208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2e143-039e-41d9-ba68-221708f584ab None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:15.658248 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ecc36-298c-48ac-bd1c-706da5d81684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:15.659360 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ecc36-298c-48ac-bd1c-706da5d81684 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:15.659712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ecc36-298c-48ac-bd1c-706da5d81684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:15.660202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ecc36-298c-48ac-bd1c-706da5d81684 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:15.661028 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ecc36-298c-48ac-bd1c-706da5d81684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:15.661448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ecc36-298c-48ac-bd1c-706da5d81684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:15.663359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ecc36-298c-48ac-bd1c-706da5d81684 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:15.684722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdf6c5-f09a-4fe3-b21b-330471d824de None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:15.691443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 966/1909] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:40:15.696026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdf6c5-f09a-4fe3-b21b-330471d824de None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:15.701738 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9647f00-4087-44b9-83db-073356bb89d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:15.702565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9647f00-4087-44b9-83db-073356bb89d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:15.702941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9647f00-4087-44b9-83db-073356bb89d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:15.703313 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9647f00-4087-44b9-83db-073356bb89d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:15.704273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9647f00-4087-44b9-83db-073356bb89d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:15.706418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9647f00-4087-44b9-83db-073356bb89d4 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:15.804569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ecc36-298c-48ac-bd1c-706da5d81684 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:15.813580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 944/1910] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:15 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:15.832491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ecc36-298c-48ac-bd1c-706da5d81684 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:15.959814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51ecb8-63e5-4f29-9070-b73b98222c04 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:15.960171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51ecb8-63e5-4f29-9070-b73b98222c04 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:15.960510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51ecb8-63e5-4f29-9070-b73b98222c04 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:15.962190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 945/1911] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:40:15.972790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23492346-14d5-4891-89a9-ccfc600f9062 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:15.973318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23492346-14d5-4891-89a9-ccfc600f9062 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:15.973760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23492346-14d5-4891-89a9-ccfc600f9062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:15.987362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67dcb3-61e6-446b-af77-4308cbc95597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:15.987980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67dcb3-61e6-446b-af77-4308cbc95597 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:15.988403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67dcb3-61e6-446b-af77-4308cbc95597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:15.988774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67dcb3-61e6-446b-af77-4308cbc95597 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:15.989704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67dcb3-61e6-446b-af77-4308cbc95597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:15.992376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67dcb3-61e6-446b-af77-4308cbc95597 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:16.002636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-23492346-14d5-4891-89a9-ccfc600f9062 None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:16.004500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-23492346-14d5-4891-89a9-ccfc600f9062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:16.023425 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 967/1912] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:15 2022] POST /identity/v3/users => generated 288 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:16.026935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67dcb3-61e6-446b-af77-4308cbc95597 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:16.039596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67dcb3-61e6-446b-af77-4308cbc95597 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:16.045040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84189a7f-beb7-42f3-80f8-e6a973b85548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:16.045896 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84189a7f-beb7-42f3-80f8-e6a973b85548 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.046386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84189a7f-beb7-42f3-80f8-e6a973b85548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.046482 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84189a7f-beb7-42f3-80f8-e6a973b85548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.047203 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84189a7f-beb7-42f3-80f8-e6a973b85548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:16.048798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84189a7f-beb7-42f3-80f8-e6a973b85548 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:16.061944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 968/1913] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:16.064975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84189a7f-beb7-42f3-80f8-e6a973b85548 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:16.073835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:16.074654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84189a7f-beb7-42f3-80f8-e6a973b85548 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:16.081177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b0fb0-2446-4fe4-a786-fea3485f5d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:16.081855 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b0fb0-2446-4fe4-a786-fea3485f5d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.082165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b0fb0-2446-4fe4-a786-fea3485f5d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.082438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b0fb0-2446-4fe4-a786-fea3485f5d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.083153 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b0fb0-2446-4fe4-a786-fea3485f5d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:16.084727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b0fb0-2446-4fe4-a786-fea3485f5d45 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:16.092820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 946/1914] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:40:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:16.095636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 969/1915] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:16.099515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23492346-14d5-4891-89a9-ccfc600f9062 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:16.105348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b0fb0-2446-4fe4-a786-fea3485f5d45 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:16.107105 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6d1b0fb0-2446-4fe4-a786-fea3485f5d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:16.110074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23492346-14d5-4891-89a9-ccfc600f9062 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:16.114730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f1397-3cd4-4a38-a48a-4a7a1c746ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:16.115612 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f1397-3cd4-4a38-a48a-4a7a1c746ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.115890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f1397-3cd4-4a38-a48a-4a7a1c746ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.116162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f1397-3cd4-4a38-a48a-4a7a1c746ef3 admin admin] PATH_INFO: `/v3/projects/5ffbff08339f48d7850713e6548004ce/users/5facc12972f9498ea64b8b7396ef7318/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.126325 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b0fb0-2446-4fe4-a786-fea3485f5d45 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:16.133458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff276789-dfc9-4b09-bf37-d667f6d2b6ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:16.134165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff276789-dfc9-4b09-bf37-d667f6d2b6ea service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.134457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff276789-dfc9-4b09-bf37-d667f6d2b6ea service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.134741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff276789-dfc9-4b09-bf37-d667f6d2b6ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.142731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f1397-3cd4-4a38-a48a-4a7a1c746ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ffbff08339f48d7850713e6548004ce, user_id=5facc12972f9498ea64b8b7396ef7318, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:16.145552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f1397-3cd4-4a38-a48a-4a7a1c746ef3 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:16.147803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff276789-dfc9-4b09-bf37-d667f6d2b6ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:16.149387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff276789-dfc9-4b09-bf37-d667f6d2b6ea service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:16.160516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 970/1916] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:16.160745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 947/1917] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:16 2022] PUT /identity/v3/projects/5ffbff08339f48d7850713e6548004ce/users/5facc12972f9498ea64b8b7396ef7318/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:16.171418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf30502-2d3b-4854-bcaa-09177d68c973 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.171762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf30502-2d3b-4854-bcaa-09177d68c973 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.172052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf30502-2d3b-4854-bcaa-09177d68c973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.202438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9cf30502-2d3b-4854-bcaa-09177d68c973 None None] MFA Rules not processed for user `5facc12972f9498ea64b8b7396ef7318`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:16.203645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9cf30502-2d3b-4854-bcaa-09177d68c973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:16.239355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:16.272948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 971/1918] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:16.277383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f1397-3cd4-4a38-a48a-4a7a1c746ef3 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:16.343262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f1397-3cd4-4a38-a48a-4a7a1c746ef3 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:16.348265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e223cf8-8547-4ddd-922f-5b0b2d107770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:16.349060 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e223cf8-8547-4ddd-922f-5b0b2d107770 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.349354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e223cf8-8547-4ddd-922f-5b0b2d107770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.349660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e223cf8-8547-4ddd-922f-5b0b2d107770 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.355374 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e223cf8-8547-4ddd-922f-5b0b2d107770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:16.357344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e223cf8-8547-4ddd-922f-5b0b2d107770 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:16.414260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 948/1919] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:16 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:16.416866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf30502-2d3b-4854-bcaa-09177d68c973 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:16.432153 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf30502-2d3b-4854-bcaa-09177d68c973 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:16.510458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0be43-e396-4ff9-a518-aaf2635bda63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:16.511168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0be43-e396-4ff9-a518-aaf2635bda63 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.511455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0be43-e396-4ff9-a518-aaf2635bda63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.511733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0be43-e396-4ff9-a518-aaf2635bda63 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.512462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0be43-e396-4ff9-a518-aaf2635bda63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:16.515059 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0be43-e396-4ff9-a518-aaf2635bda63 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:16.541880 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 972/1920] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:16 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:16.543102 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e223cf8-8547-4ddd-922f-5b0b2d107770 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:16.554136 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e223cf8-8547-4ddd-922f-5b0b2d107770 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:16.566104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a0df4-b5e4-4ab5-b2f3-5d99db08dfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:16.566104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a0df4-b5e4-4ab5-b2f3-5d99db08dfec admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.566104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a0df4-b5e4-4ab5-b2f3-5d99db08dfec admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.566104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a0df4-b5e4-4ab5-b2f3-5d99db08dfec admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.566491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a0df4-b5e4-4ab5-b2f3-5d99db08dfec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:16.566491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a0df4-b5e4-4ab5-b2f3-5d99db08dfec admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:16.576740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 949/1921] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:16.580525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0be43-e396-4ff9-a518-aaf2635bda63 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:16.590581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0be43-e396-4ff9-a518-aaf2635bda63 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:16.595265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62546d-43cf-4aba-8a7b-fd9f2f1e3e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:16.596159 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62546d-43cf-4aba-8a7b-fd9f2f1e3e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.596442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62546d-43cf-4aba-8a7b-fd9f2f1e3e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.596716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62546d-43cf-4aba-8a7b-fd9f2f1e3e46 admin admin] PATH_INFO: `/v3/projects/ae1af734309f4d06a0852f01b862dc08/users/317d88b6f46743af97cc7343f4c0718e/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.611995 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62546d-43cf-4aba-8a7b-fd9f2f1e3e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae1af734309f4d06a0852f01b862dc08, user_id=317d88b6f46743af97cc7343f4c0718e, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:16.614927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62546d-43cf-4aba-8a7b-fd9f2f1e3e46 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:16.627970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 973/1922] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:16 2022] PUT /identity/v3/projects/ae1af734309f4d06a0852f01b862dc08/users/317d88b6f46743af97cc7343f4c0718e/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:16.631882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a0df4-b5e4-4ab5-b2f3-5d99db08dfec admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:16.642399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a0df4-b5e4-4ab5-b2f3-5d99db08dfec admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:16.698153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beabd15-24e0-47e8-9dc9-ee722c8e2c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:16.698822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beabd15-24e0-47e8-9dc9-ee722c8e2c8a admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.699108 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beabd15-24e0-47e8-9dc9-ee722c8e2c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.699385 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beabd15-24e0-47e8-9dc9-ee722c8e2c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.700133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beabd15-24e0-47e8-9dc9-ee722c8e2c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:16.701767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beabd15-24e0-47e8-9dc9-ee722c8e2c8a admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:16.711266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 950/1923] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:16.715438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62546d-43cf-4aba-8a7b-fd9f2f1e3e46 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:16.725730 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62546d-43cf-4aba-8a7b-fd9f2f1e3e46 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:16.730372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1c728-841e-4d60-96de-93812438852a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:16.731249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1c728-841e-4d60-96de-93812438852a admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.731532 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1c728-841e-4d60-96de-93812438852a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.731807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1c728-841e-4d60-96de-93812438852a admin admin] PATH_INFO: `/v3/projects/ae1af734309f4d06a0852f01b862dc08/users/317d88b6f46743af97cc7343f4c0718e/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.734366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1c728-841e-4d60-96de-93812438852a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae1af734309f4d06a0852f01b862dc08, user_id=317d88b6f46743af97cc7343f4c0718e, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:16.737343 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1c728-841e-4d60-96de-93812438852a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:16.747531 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 974/1924] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:16 2022] PUT /identity/v3/projects/ae1af734309f4d06a0852f01b862dc08/users/317d88b6f46743af97cc7343f4c0718e/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:16.759432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e0a36-2335-484a-89a3-fc6bf6de1ae2 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.759722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e0a36-2335-484a-89a3-fc6bf6de1ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.760034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e0a36-2335-484a-89a3-fc6bf6de1ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.786989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-830e0a36-2335-484a-89a3-fc6bf6de1ae2 None None] MFA Rules not processed for user `317d88b6f46743af97cc7343f4c0718e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:16.788442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-830e0a36-2335-484a-89a3-fc6bf6de1ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:16.815534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:16.856078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 951/1925] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:16 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:16.867717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1c728-841e-4d60-96de-93812438852a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:16.869394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-eef1c728-841e-4d60-96de-93812438852a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:16.925064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1c728-841e-4d60-96de-93812438852a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:16.928679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b9579-ceb4-4fb8-b5e6-431ae0a564eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:16.929311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b9579-ceb4-4fb8-b5e6-431ae0a564eb service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:16.929594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b9579-ceb4-4fb8-b5e6-431ae0a564eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:16.929871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b9579-ceb4-4fb8-b5e6-431ae0a564eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:16.971170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:16.989682 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b9579-ceb4-4fb8-b5e6-431ae0a564eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:16.991181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b9579-ceb4-4fb8-b5e6-431ae0a564eb service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:17.000930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 975/1926] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:26.853214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=17 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:26.996068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=9 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:27.988542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e0a36-2335-484a-89a3-fc6bf6de1ae2 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:28.053038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e0a36-2335-484a-89a3-fc6bf6de1ae2 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:28.057498 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:28.058392 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:28.058676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:28.058951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] PATH_INFO: `/v3/users/5facc12972f9498ea64b8b7396ef7318` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:28.060165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5facc12972f9498ea64b8b7396ef7318)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:28.062118 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:28.109724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5facc12972f9498ea64b8b7396ef7318'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.110506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5facc12972f9498ea64b8b7396ef7318'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.111133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5facc12972f9498ea64b8b7396ef7318'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.118184 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5facc12972f9498ea64b8b7396ef7318'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.126747 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5facc12972f9498ea64b8b7396ef7318'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.145597 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 952/1927] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:27 2022] DELETE /identity/v3/users/5facc12972f9498ea64b8b7396ef7318 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:28.150310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60dec0-e8a8-40a7-86c1-7b9eedc818a7 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:28.150807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60dec0-e8a8-40a7-86c1-7b9eedc818a7 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:28.150984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60dec0-e8a8-40a7-86c1-7b9eedc818a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:28.180825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4b60dec0-e8a8-40a7-86c1-7b9eedc818a7 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:28.182463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4b60dec0-e8a8-40a7-86c1-7b9eedc818a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:28.244721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 976/1928] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:28.255474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:28.258393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:28.317282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2a4b1-93d3-41b8-b050-c36fa4da4ea8 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:28.320867 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbc357-dd0f-49b3-ae07-7d6d6fa21f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:28.321511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbc357-dd0f-49b3-ae07-7d6d6fa21f78 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:28.321797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbc357-dd0f-49b3-ae07-7d6d6fa21f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:28.322104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbc357-dd0f-49b3-ae07-7d6d6fa21f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:28.332088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60dec0-e8a8-40a7-86c1-7b9eedc818a7 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:28.332509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbc357-dd0f-49b3-ae07-7d6d6fa21f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:28.333462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4b60dec0-e8a8-40a7-86c1-7b9eedc818a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:28.333964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbc357-dd0f-49b3-ae07-7d6d6fa21f78 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:28.343839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 953/1929] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:28.347502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60dec0-e8a8-40a7-86c1-7b9eedc818a7 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:28.352376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:28.353268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:28.353556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:28.353834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] PATH_INFO: `/v3/users/a20085e2d5e945dcbc8cc1c121f3bde2` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:28.355078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20085e2d5e945dcbc8cc1c121f3bde2)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:28.357110 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:28.421571 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20085e2d5e945dcbc8cc1c121f3bde2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.431501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20085e2d5e945dcbc8cc1c121f3bde2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.433002 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20085e2d5e945dcbc8cc1c121f3bde2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.442696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20085e2d5e945dcbc8cc1c121f3bde2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.443510 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20085e2d5e945dcbc8cc1c121f3bde2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.462716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 977/1930] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:28 2022] DELETE /identity/v3/users/a20085e2d5e945dcbc8cc1c121f3bde2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:28.467574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3a252-f987-4413-9edc-cd85c0a4d9a5 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:28.467850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3a252-f987-4413-9edc-cd85c0a4d9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:28.468182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3a252-f987-4413-9edc-cd85c0a4d9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:28.495186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-58c3a252-f987-4413-9edc-cd85c0a4d9a5 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:28.496422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-58c3a252-f987-4413-9edc-cd85c0a4d9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:28.523030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:28.571416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 954/1931] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:28.581720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:28.583310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:28.641609 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307edcc-e2ca-45b5-b482-d9aa78d55dd6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:28.645255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d031b7-d9e4-45b1-8c6f-5da0a77a9a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:28.645907 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d031b7-d9e4-45b1-8c6f-5da0a77a9a96 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:28.646216 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d031b7-d9e4-45b1-8c6f-5da0a77a9a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:28.646490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d031b7-d9e4-45b1-8c6f-5da0a77a9a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:28.655698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d031b7-d9e4-45b1-8c6f-5da0a77a9a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:28.657203 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d031b7-d9e4-45b1-8c6f-5da0a77a9a96 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:28.666964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 978/1932] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:28.725522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3a252-f987-4413-9edc-cd85c0a4d9a5 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:28.727012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-58c3a252-f987-4413-9edc-cd85c0a4d9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:28.741190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3a252-f987-4413-9edc-cd85c0a4d9a5 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:28.754039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:28.755021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:28.756192 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:28.756762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] PATH_INFO: `/v3/projects/5ffbff08339f48d7850713e6548004ce` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:28.758563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ffbff08339f48d7850713e6548004ce)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:28.760663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:28.852897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ffbff08339f48d7850713e6548004ce', 'request_id': 'req-37998d8d-02a0-40d5-9aac-5a9edbb8afde'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.863813 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ffbff08339f48d7850713e6548004ce', 'request_id': 'req-37998d8d-02a0-40d5-9aac-5a9edbb8afde'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.871026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ffbff08339f48d7850713e6548004ce', 'request_id': 'req-37998d8d-02a0-40d5-9aac-5a9edbb8afde'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.896206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] The token cache is being invalidate because project 5ffbff08339f48d7850713e6548004ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:28.896506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.897795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:28.903475 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 955/1933] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:28 2022] DELETE /identity/v3/projects/5ffbff08339f48d7850713e6548004ce => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:28.907770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d031b7-d9e4-45b1-8c6f-5da0a77a9a96 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:28.922730 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d031b7-d9e4-45b1-8c6f-5da0a77a9a96 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:28.954636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:29.048315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:29.050340 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:29.052102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:29.052662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] PATH_INFO: `/v3/users/317d88b6f46743af97cc7343f4c0718e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:29.054359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] RBAC: Authorizing `identity:delete_user(user_id=317d88b6f46743af97cc7343f4c0718e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:29.057380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:29.159826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '317d88b6f46743af97cc7343f4c0718e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.167397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:29.170487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:29.179415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '317d88b6f46743af97cc7343f4c0718e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.180226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '317d88b6f46743af97cc7343f4c0718e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.197476 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '317d88b6f46743af97cc7343f4c0718e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.198376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '317d88b6f46743af97cc7343f4c0718e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.208934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37998d8d-02a0-40d5-9aac-5a9edbb8afde admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:29.215098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:29.216934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:29.217179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:29.217525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] PATH_INFO: `/v3/projects/88f7790fae3846db911734a468a664ab` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:29.218872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f7790fae3846db911734a468a664ab)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:29.220918 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:29.227121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 979/1934] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:28 2022] DELETE /identity/v3/users/317d88b6f46743af97cc7343f4c0718e => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:29.290037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f7790fae3846db911734a468a664ab', 'request_id': 'req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.305138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f7790fae3846db911734a468a664ab', 'request_id': 'req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.317782 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f7790fae3846db911734a468a664ab', 'request_id': 'req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.338148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] The token cache is being invalidate because project 88f7790fae3846db911734a468a664ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:29.338440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.339083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.344440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 956/1935] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:29 2022] DELETE /identity/v3/projects/88f7790fae3846db911734a468a664ab => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:29.345654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:29.347277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:29.436444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6246a2-3bf7-472a-a061-e29aa381e409 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:29.482640 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:29.506096 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7a3fd-9a2c-4a21-b2a7-83bb07748322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:29.507129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7a3fd-9a2c-4a21-b2a7-83bb07748322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:29.507411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7a3fd-9a2c-4a21-b2a7-83bb07748322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:29.507729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7a3fd-9a2c-4a21-b2a7-83bb07748322 admin admin] PATH_INFO: `/v3/users/4658ee5cc14e418a91e7dd4226f23693` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:29.509121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7a3fd-9a2c-4a21-b2a7-83bb07748322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4658ee5cc14e418a91e7dd4226f23693)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:29.511525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7a3fd-9a2c-4a21-b2a7-83bb07748322 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:29.571304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-15f7a3fd-9a2c-4a21-b2a7-83bb07748322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4658ee5cc14e418a91e7dd4226f23693'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.580010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-15f7a3fd-9a2c-4a21-b2a7-83bb07748322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4658ee5cc14e418a91e7dd4226f23693'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.580732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-15f7a3fd-9a2c-4a21-b2a7-83bb07748322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4658ee5cc14e418a91e7dd4226f23693'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.589481 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-15f7a3fd-9a2c-4a21-b2a7-83bb07748322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4658ee5cc14e418a91e7dd4226f23693'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.590187 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-15f7a3fd-9a2c-4a21-b2a7-83bb07748322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4658ee5cc14e418a91e7dd4226f23693'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.613833 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 980/1936] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:29 2022] DELETE /identity/v3/users/4658ee5cc14e418a91e7dd4226f23693 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:29.638472 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:29.639685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:29.655645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7285e8-ddd0-46a0-93c1-9202e07fdd6f admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:29.694572 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:29.758051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4499e9a-26d9-40ca-add7-60675fa3c842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:29.759164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4499e9a-26d9-40ca-add7-60675fa3c842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:29.759559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4499e9a-26d9-40ca-add7-60675fa3c842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:29.759909 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4499e9a-26d9-40ca-add7-60675fa3c842 admin admin] PATH_INFO: `/v3/projects/ae1af734309f4d06a0852f01b862dc08` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:29.762780 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4499e9a-26d9-40ca-add7-60675fa3c842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae1af734309f4d06a0852f01b862dc08)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:29.765403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4499e9a-26d9-40ca-add7-60675fa3c842 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:29.818329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b4499e9a-26d9-40ca-add7-60675fa3c842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae1af734309f4d06a0852f01b862dc08', 'request_id': 'req-b4499e9a-26d9-40ca-add7-60675fa3c842'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.826331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b4499e9a-26d9-40ca-add7-60675fa3c842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae1af734309f4d06a0852f01b862dc08', 'request_id': 'req-b4499e9a-26d9-40ca-add7-60675fa3c842'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.832021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b4499e9a-26d9-40ca-add7-60675fa3c842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae1af734309f4d06a0852f01b862dc08', 'request_id': 'req-b4499e9a-26d9-40ca-add7-60675fa3c842'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.844811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b4499e9a-26d9-40ca-add7-60675fa3c842 admin admin] The token cache is being invalidate because project ae1af734309f4d06a0852f01b862dc08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:29.845100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b4499e9a-26d9-40ca-add7-60675fa3c842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.845738 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b4499e9a-26d9-40ca-add7-60675fa3c842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:29.847317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 957/1937] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:29 2022] DELETE /identity/v3/projects/ae1af734309f4d06a0852f01b862dc08 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:29.852386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e12b9-072e-4ac8-ae3b-4ad67b001fb8 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:29.852681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e12b9-072e-4ac8-ae3b-4ad67b001fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:29.852984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e12b9-072e-4ac8-ae3b-4ad67b001fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:29.876033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f46e12b9-072e-4ac8-ae3b-4ad67b001fb8 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:29.877240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f46e12b9-072e-4ac8-ae3b-4ad67b001fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:29.942480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 981/1938] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:29.950147 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343ceeb-8903-420e-8085-50dcf1f8bf2f None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:29.951366 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343ceeb-8903-420e-8085-50dcf1f8bf2f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:29.952049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343ceeb-8903-420e-8085-50dcf1f8bf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:29.973292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f343ceeb-8903-420e-8085-50dcf1f8bf2f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:29.974628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f343ceeb-8903-420e-8085-50dcf1f8bf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:29.983781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 958/1939] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:29.988399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e12b9-072e-4ac8-ae3b-4ad67b001fb8 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:29.991865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343ceeb-8903-420e-8085-50dcf1f8bf2f None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:29.993270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f343ceeb-8903-420e-8085-50dcf1f8bf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:29.999205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e12b9-072e-4ac8-ae3b-4ad67b001fb8 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.003879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a6e56-c35c-46e1-a20c-abb40aa19bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.004557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a6e56-c35c-46e1-a20c-abb40aa19bbc admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.004843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a6e56-c35c-46e1-a20c-abb40aa19bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.005119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a6e56-c35c-46e1-a20c-abb40aa19bbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.006031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a6e56-c35c-46e1-a20c-abb40aa19bbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:30.006342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a6e56-c35c-46e1-a20c-abb40aa19bbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.007884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343ceeb-8903-420e-8085-50dcf1f8bf2f None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.008011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a6e56-c35c-46e1-a20c-abb40aa19bbc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.014422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.015334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.015655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.016495 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1 admin admin] PATH_INFO: `/v3/projects/4adc50f6a89f4792a617c8ce265219f4` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.018270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4adc50f6a89f4792a617c8ce265219f4)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.020626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.023291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 982/1940] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:40:30.027168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a6e56-c35c-46e1-a20c-abb40aa19bbc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.036929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a6e56-c35c-46e1-a20c-abb40aa19bbc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.041682 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabc3bd-8228-4511-85e5-73f2026bb613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.042364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabc3bd-8228-4511-85e5-73f2026bb613 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.042651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabc3bd-8228-4511-85e5-73f2026bb613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.042924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabc3bd-8228-4511-85e5-73f2026bb613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.043729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabc3bd-8228-4511-85e5-73f2026bb613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.045682 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabc3bd-8228-4511-85e5-73f2026bb613 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.072631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4adc50f6a89f4792a617c8ce265219f4', 'request_id': 'req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:30.074068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 983/1941] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:30 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:30.078481 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabc3bd-8228-4511-85e5-73f2026bb613 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.087771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4adc50f6a89f4792a617c8ce265219f4', 'request_id': 'req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:30.094444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4adc50f6a89f4792a617c8ce265219f4', 'request_id': 'req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:30.110357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1 admin admin] The token cache is being invalidate because project 4adc50f6a89f4792a617c8ce265219f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:30.110835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:30.111734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a2a11b7a-22d6-4193-afa4-c4a2ee0d38b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:30.113265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:30.113608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 959/1942] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:29 2022] DELETE /identity/v3/projects/4adc50f6a89f4792a617c8ce265219f4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:30.118460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60489d-3091-4d02-a730-d5d8cfadfd93 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.119093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60489d-3091-4d02-a730-d5d8cfadfd93 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.119389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60489d-3091-4d02-a730-d5d8cfadfd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.142169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-7c60489d-3091-4d02-a730-d5d8cfadfd93 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:30.143374 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7c60489d-3091-4d02-a730-d5d8cfadfd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:30.162671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabc3bd-8228-4511-85e5-73f2026bb613 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.168344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37b765-d1cb-4b3a-89ae-63d68ac4f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.170104 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37b765-d1cb-4b3a-89ae-63d68ac4f869 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.170562 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37b765-d1cb-4b3a-89ae-63d68ac4f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.171001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37b765-d1cb-4b3a-89ae-63d68ac4f869 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.171912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37b765-d1cb-4b3a-89ae-63d68ac4f869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.174077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37b765-d1cb-4b3a-89ae-63d68ac4f869 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.186607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:30.199081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 984/1943] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:30 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:30.203592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37b765-d1cb-4b3a-89ae-63d68ac4f869 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.205712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4a37b765-d1cb-4b3a-89ae-63d68ac4f869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:30.219613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37b765-d1cb-4b3a-89ae-63d68ac4f869 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.228144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a9fbb-2cf1-4b5d-b9f0-6f255158cdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.228144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a9fbb-2cf1-4b5d-b9f0-6f255158cdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.228144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a9fbb-2cf1-4b5d-b9f0-6f255158cdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.228144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a9fbb-2cf1-4b5d-b9f0-6f255158cdf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.228577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 960/1944] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:30.228667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a9fbb-2cf1-4b5d-b9f0-6f255158cdf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.228667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a9fbb-2cf1-4b5d-b9f0-6f255158cdf8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.231379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbdee4-dd1f-4ff4-9834-de4de65670ad None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.232198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbdee4-dd1f-4ff4-9834-de4de65670ad None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.232478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbdee4-dd1f-4ff4-9834-de4de65670ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.239080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 985/1945] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:30.243027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a9fbb-2cf1-4b5d-b9f0-6f255158cdf8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.255080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a9fbb-2cf1-4b5d-b9f0-6f255158cdf8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.260536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ac24e-aa74-4469-a406-76d71b70729f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.261152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ac24e-aa74-4469-a406-76d71b70729f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.261456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ac24e-aa74-4469-a406-76d71b70729f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.261699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ac24e-aa74-4469-a406-76d71b70729f admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.265437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ac24e-aa74-4469-a406-76d71b70729f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.265437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ac24e-aa74-4469-a406-76d71b70729f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.272810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-6abbdee4-dd1f-4ff4-9834-de4de65670ad None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:30.274121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6abbdee4-dd1f-4ff4-9834-de4de65670ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:30.285479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 961/1946] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:30.294650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 986/1947] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:30.296241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbdee4-dd1f-4ff4-9834-de4de65670ad None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.298973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ac24e-aa74-4469-a406-76d71b70729f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.306019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbdee4-dd1f-4ff4-9834-de4de65670ad None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.311519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cba6d-43a6-444c-9ab6-cd7e44e4da0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.311960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ac24e-aa74-4469-a406-76d71b70729f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.312335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cba6d-43a6-444c-9ab6-cd7e44e4da0b admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.312624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cba6d-43a6-444c-9ab6-cd7e44e4da0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.312898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cba6d-43a6-444c-9ab6-cd7e44e4da0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.313624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222cba6d-43a6-444c-9ab6-cd7e44e4da0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:30.313941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222cba6d-43a6-444c-9ab6-cd7e44e4da0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.315569 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222cba6d-43a6-444c-9ab6-cd7e44e4da0b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.317506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee65d4e-19a6-4e83-8db3-ca7af91d0b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.318883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee65d4e-19a6-4e83-8db3-ca7af91d0b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.319166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee65d4e-19a6-4e83-8db3-ca7af91d0b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.319444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee65d4e-19a6-4e83-8db3-ca7af91d0b9c admin admin] PATH_INFO: `/v3/projects/5d2fa8874dab4d3bbc87161b540ffae9/users/0f4376954dbd4402ae5a990f1a56baf3/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.331054 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 962/1948] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:40:30.332684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cba6d-43a6-444c-9ab6-cd7e44e4da0b admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.341557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee65d4e-19a6-4e83-8db3-ca7af91d0b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d2fa8874dab4d3bbc87161b540ffae9, user_id=0f4376954dbd4402ae5a990f1a56baf3, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.343261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cba6d-43a6-444c-9ab6-cd7e44e4da0b admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.344360 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee65d4e-19a6-4e83-8db3-ca7af91d0b9c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.348169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d14c6e-5cf3-40c1-b272-4d738146be6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.348983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d14c6e-5cf3-40c1-b272-4d738146be6c admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.349276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d14c6e-5cf3-40c1-b272-4d738146be6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.350219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d14c6e-5cf3-40c1-b272-4d738146be6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.351078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d14c6e-5cf3-40c1-b272-4d738146be6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.353125 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d14c6e-5cf3-40c1-b272-4d738146be6c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.362000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 987/1949] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:30 2022] PUT /identity/v3/projects/5d2fa8874dab4d3bbc87161b540ffae9/users/0f4376954dbd4402ae5a990f1a56baf3/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:30.367589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daeb449-fa12-449a-91e0-30c280b347b5 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.368155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daeb449-fa12-449a-91e0-30c280b347b5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.368155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daeb449-fa12-449a-91e0-30c280b347b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.390425 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3daeb449-fa12-449a-91e0-30c280b347b5 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:30.391744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3daeb449-fa12-449a-91e0-30c280b347b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:30.394150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 963/1950] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:30 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:30.395734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d14c6e-5cf3-40c1-b272-4d738146be6c admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.463959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 988/1951] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:30.464959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daeb449-fa12-449a-91e0-30c280b347b5 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.466833 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3daeb449-fa12-449a-91e0-30c280b347b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:30.469810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d14c6e-5cf3-40c1-b272-4d738146be6c admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.474905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6bc72-0a26-48df-9f03-4c70ce944307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.475645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6bc72-0a26-48df-9f03-4c70ce944307 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.475930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6bc72-0a26-48df-9f03-4c70ce944307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.476204 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6bc72-0a26-48df-9f03-4c70ce944307 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.476894 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6bc72-0a26-48df-9f03-4c70ce944307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.478461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daeb449-fa12-449a-91e0-30c280b347b5 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.478862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6bc72-0a26-48df-9f03-4c70ce944307 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.483752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.484872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.485148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.485418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] PATH_INFO: `/v3/users/cb0b87ea736144f0b41c020a6c0a0eef` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.486623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb0b87ea736144f0b41c020a6c0a0eef)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.488517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.505272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 964/1952] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:30 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:30.505272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6bc72-0a26-48df-9f03-4c70ce944307 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.505272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-bbf6bc72-0a26-48df-9f03-4c70ce944307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:30.545858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb0b87ea736144f0b41c020a6c0a0eef'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:30.554431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb0b87ea736144f0b41c020a6c0a0eef'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:30.555070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb0b87ea736144f0b41c020a6c0a0eef'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:30.563257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb0b87ea736144f0b41c020a6c0a0eef'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:30.564087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb0b87ea736144f0b41c020a6c0a0eef'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:30.567162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:30.570232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6bc72-0a26-48df-9f03-4c70ce944307 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.574268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b80d2-b704-4f5a-a71d-e7b98c9ab8ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.575065 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b80d2-b704-4f5a-a71d-e7b98c9ab8ee service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.575457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b80d2-b704-4f5a-a71d-e7b98c9ab8ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.575650 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b80d2-b704-4f5a-a71d-e7b98c9ab8ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.577762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-125b80d2-b704-4f5a-a71d-e7b98c9ab8ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:30.590348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b80d2-b704-4f5a-a71d-e7b98c9ab8ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.594363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 989/1953] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:30 2022] DELETE /identity/v3/users/cb0b87ea736144f0b41c020a6c0a0eef => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:30.594363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.594597 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b80d2-b704-4f5a-a71d-e7b98c9ab8ee service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.594674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:30.603629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 965/1954] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:30.604901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54ea4d-968e-407c-b791-7bbb00833035 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.605205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54ea4d-968e-407c-b791-7bbb00833035 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.605476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54ea4d-968e-407c-b791-7bbb00833035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.626993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-6e54ea4d-968e-407c-b791-7bbb00833035 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:30.628207 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6e54ea4d-968e-407c-b791-7bbb00833035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:30.654148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:30.674551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58aa739-301f-407b-8659-8af7c4ff936a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.679463 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd6633-dd56-4bf7-a44d-38808281a319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.680140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd6633-dd56-4bf7-a44d-38808281a319 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.680421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd6633-dd56-4bf7-a44d-38808281a319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.680707 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd6633-dd56-4bf7-a44d-38808281a319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.681436 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd6633-dd56-4bf7-a44d-38808281a319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.683099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd6633-dd56-4bf7-a44d-38808281a319 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.694070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 990/1955] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:30.698784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd6633-dd56-4bf7-a44d-38808281a319 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.700584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 966/1956] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:30.710810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd6633-dd56-4bf7-a44d-38808281a319 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.712502 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54ea4d-968e-407c-b791-7bbb00833035 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.714065 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6e54ea4d-968e-407c-b791-7bbb00833035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:30.716909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407a355-8153-46dc-a87f-8152f5223ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.717688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407a355-8153-46dc-a87f-8152f5223ede admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.718051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407a355-8153-46dc-a87f-8152f5223ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.718333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407a355-8153-46dc-a87f-8152f5223ede admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.719069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407a355-8153-46dc-a87f-8152f5223ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.720963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407a355-8153-46dc-a87f-8152f5223ede admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.734776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 991/1957] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:30.739252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407a355-8153-46dc-a87f-8152f5223ede admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.757788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407a355-8153-46dc-a87f-8152f5223ede admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.764662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fa5cd-da10-42b7-8d49-795e29f75e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.765708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa5cd-da10-42b7-8d49-795e29f75e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.766048 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa5cd-da10-42b7-8d49-795e29f75e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.766363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa5cd-da10-42b7-8d49-795e29f75e48 admin admin] PATH_INFO: `/v3/projects/1d46d4698d3d43c2a15a6b40f08090af/users/37d3ed0e5f0f43ee8a1404b7ab856cd9/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.788024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fa5cd-da10-42b7-8d49-795e29f75e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d46d4698d3d43c2a15a6b40f08090af, user_id=37d3ed0e5f0f43ee8a1404b7ab856cd9, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.790616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54ea4d-968e-407c-b791-7bbb00833035 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:30.791576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fa5cd-da10-42b7-8d49-795e29f75e48 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.795965 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0545f33-9e98-404e-9a0f-d9a19ef4a62f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:30.796926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0545f33-9e98-404e-9a0f-d9a19ef4a62f service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.797296 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0545f33-9e98-404e-9a0f-d9a19ef4a62f service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.797619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0545f33-9e98-404e-9a0f-d9a19ef4a62f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.808160 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0545f33-9e98-404e-9a0f-d9a19ef4a62f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:30.809619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0545f33-9e98-404e-9a0f-d9a19ef4a62f service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:30.811582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 992/1958] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:30 2022] PUT /identity/v3/projects/1d46d4698d3d43c2a15a6b40f08090af/users/37d3ed0e5f0f43ee8a1404b7ab856cd9/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:30.824240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5031b-a523-4c34-acd9-9dbbaf877dd1 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:30.824577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5031b-a523-4c34-acd9-9dbbaf877dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:30.824835 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5031b-a523-4c34-acd9-9dbbaf877dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:30.828630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 967/1959] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:40:30.859204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-11e5031b-a523-4c34-acd9-9dbbaf877dd1 None None] MFA Rules not processed for user `37d3ed0e5f0f43ee8a1404b7ab856cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:30.860813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-11e5031b-a523-4c34-acd9-9dbbaf877dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:30.946682 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 993/1960] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:30.955651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0545f33-9e98-404e-9a0f-d9a19ef4a62f service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:30.961266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b0545f33-9e98-404e-9a0f-d9a19ef4a62f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:31.093152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0545f33-9e98-404e-9a0f-d9a19ef4a62f service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:31.193220 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a913e-7277-47a8-902c-99293565450c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b07dcd8c0de4d1694d4363520726df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:31.194091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a913e-7277-47a8-902c-99293565450c service glance] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:31.194447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a913e-7277-47a8-902c-99293565450c service glance] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:31.194793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a913e-7277-47a8-902c-99293565450c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:31.207114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a913e-7277-47a8-902c-99293565450c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:31.208457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a913e-7277-47a8-902c-99293565450c service glance] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:31.218667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 968/1961] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:31.313457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5031b-a523-4c34-acd9-9dbbaf877dd1 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:31.315376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-11e5031b-a523-4c34-acd9-9dbbaf877dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:31.364422 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:31.412259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5031b-a523-4c34-acd9-9dbbaf877dd1 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:31.441116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:31.442144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:31.442508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:31.442866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] PATH_INFO: `/v3/projects/2a4bed0a77f34ddaa655b9c17f29825f` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:31.444220 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4bed0a77f34ddaa655b9c17f29825f)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:31.449734 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:31.540987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead1ca0-f9a3-4551-b7ea-d9f15699ba41 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:31.541476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead1ca0-f9a3-4551-b7ea-d9f15699ba41 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:31.541975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead1ca0-f9a3-4551-b7ea-d9f15699ba41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:31.543477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4bed0a77f34ddaa655b9c17f29825f', 'request_id': 'req-cb847b33-97b3-4dc7-8b66-d275e3b227db'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:31.555430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4bed0a77f34ddaa655b9c17f29825f', 'request_id': 'req-cb847b33-97b3-4dc7-8b66-d275e3b227db'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:31.565949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:31.570859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4bed0a77f34ddaa655b9c17f29825f', 'request_id': 'req-cb847b33-97b3-4dc7-8b66-d275e3b227db'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:31.579877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-aead1ca0-f9a3-4551-b7ea-d9f15699ba41 None None] MFA Rules not processed for user `0f4376954dbd4402ae5a990f1a56baf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:31.581169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-aead1ca0-f9a3-4551-b7ea-d9f15699ba41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:31.590150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] The token cache is being invalidate because project 2a4bed0a77f34ddaa655b9c17f29825f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:31.590451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:31.591689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:31.593298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 994/1962] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:31 2022] DELETE /identity/v3/projects/2a4bed0a77f34ddaa655b9c17f29825f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:31.597352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:31.599398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:31.661259 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 969/1963] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:31.677890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead1ca0-f9a3-4551-b7ea-d9f15699ba41 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:31.679568 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-aead1ca0-f9a3-4551-b7ea-d9f15699ba41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:31.693407 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb847b33-97b3-4dc7-8b66-d275e3b227db admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:31.708965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e6ee0-edfc-4124-ac44-a5ad664270ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:31.710611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e6ee0-edfc-4124-ac44-a5ad664270ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:31.710959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e6ee0-edfc-4124-ac44-a5ad664270ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:31.711444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e6ee0-edfc-4124-ac44-a5ad664270ca admin admin] PATH_INFO: `/v3/users/56070d690cef4063a026d69d93937537` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:31.714277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e6ee0-edfc-4124-ac44-a5ad664270ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=56070d690cef4063a026d69d93937537)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:31.718787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e6ee0-edfc-4124-ac44-a5ad664270ca admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:31.749266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead1ca0-f9a3-4551-b7ea-d9f15699ba41 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:31.753630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88216da0-b48c-4c13-b864-43df031647d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:31.754556 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88216da0-b48c-4c13-b864-43df031647d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:31.755109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88216da0-b48c-4c13-b864-43df031647d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:31.755681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88216da0-b48c-4c13-b864-43df031647d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:31.768751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88216da0-b48c-4c13-b864-43df031647d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:31.770521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88216da0-b48c-4c13-b864-43df031647d9 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:31.783476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 970/1964] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:40:31.789368 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-943e6ee0-edfc-4124-ac44-a5ad664270ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56070d690cef4063a026d69d93937537'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:31.800261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-943e6ee0-edfc-4124-ac44-a5ad664270ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56070d690cef4063a026d69d93937537'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:31.801007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-943e6ee0-edfc-4124-ac44-a5ad664270ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56070d690cef4063a026d69d93937537'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:31.808566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-943e6ee0-edfc-4124-ac44-a5ad664270ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56070d690cef4063a026d69d93937537'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:31.809334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-943e6ee0-edfc-4124-ac44-a5ad664270ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56070d690cef4063a026d69d93937537'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:31.811619 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:31.833404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 995/1965] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:31 2022] DELETE /identity/v3/users/56070d690cef4063a026d69d93937537 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:32.232096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88216da0-b48c-4c13-b864-43df031647d9 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:32.246284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88216da0-b48c-4c13-b864-43df031647d9 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:32.314888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:32.315743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:32.316020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:32.316291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc admin admin] PATH_INFO: `/v3/projects/3e0c6a7972864dcd8e7d992ae196f314` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:32.317480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e0c6a7972864dcd8e7d992ae196f314)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:32.319424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:32.364762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:32.369052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e0c6a7972864dcd8e7d992ae196f314', 'request_id': 'req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:32.377428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e0c6a7972864dcd8e7d992ae196f314', 'request_id': 'req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:32.383494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e0c6a7972864dcd8e7d992ae196f314', 'request_id': 'req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:32.397305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc admin admin] The token cache is being invalidate because project 3e0c6a7972864dcd8e7d992ae196f314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:32.397885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:32.398596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-cb2d542d-cd8d-4fbf-83f9-3ea6c1197bbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:32.400245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 971/1966] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:32 2022] DELETE /identity/v3/projects/3e0c6a7972864dcd8e7d992ae196f314 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:32.405404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1020e5-3663-4cc6-905e-2a8ab2085d36 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:32.405953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1020e5-3663-4cc6-905e-2a8ab2085d36 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:32.406068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1020e5-3663-4cc6-905e-2a8ab2085d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:32.426957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6d1020e5-3663-4cc6-905e-2a8ab2085d36 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:32.428182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6d1020e5-3663-4cc6-905e-2a8ab2085d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:32.499242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 996/1967] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:32.501748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee1ac2-bbe3-48e7-98bd-4d6af59d69fb None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:32.502061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee1ac2-bbe3-48e7-98bd-4d6af59d69fb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:32.502331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee1ac2-bbe3-48e7-98bd-4d6af59d69fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:32.522491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-cdee1ac2-bbe3-48e7-98bd-4d6af59d69fb None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:32.523698 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-cdee1ac2-bbe3-48e7-98bd-4d6af59d69fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:32.532239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 972/1968] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:32.536455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1020e5-3663-4cc6-905e-2a8ab2085d36 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:32.546825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1020e5-3663-4cc6-905e-2a8ab2085d36 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:32.551605 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb2119-ef6f-4048-b111-073c45ab822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:32.552406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb2119-ef6f-4048-b111-073c45ab822a admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:32.552752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb2119-ef6f-4048-b111-073c45ab822a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:32.553098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb2119-ef6f-4048-b111-073c45ab822a admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:32.553896 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb2119-ef6f-4048-b111-073c45ab822a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:32.554293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb2119-ef6f-4048-b111-073c45ab822a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:32.555958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb2119-ef6f-4048-b111-073c45ab822a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:32.568802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 997/1969] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:40:32.573242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee1ac2-bbe3-48e7-98bd-4d6af59d69fb None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:32.583107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee1ac2-bbe3-48e7-98bd-4d6af59d69fb None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:32.588292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721c26f-150b-46c9-aa8b-cabcffffa4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:32.588956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721c26f-150b-46c9-aa8b-cabcffffa4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:32.589234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721c26f-150b-46c9-aa8b-cabcffffa4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:32.589505 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721c26f-150b-46c9-aa8b-cabcffffa4a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:32.590249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721c26f-150b-46c9-aa8b-cabcffffa4a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:32.592400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721c26f-150b-46c9-aa8b-cabcffffa4a0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:32.627971 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 973/1970] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:32 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:32.634016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb2119-ef6f-4048-b111-073c45ab822a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:32.644042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb2119-ef6f-4048-b111-073c45ab822a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:32.708185 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251f840-5a5c-492b-9ec3-8ffdf80bcdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:32.708855 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251f840-5a5c-492b-9ec3-8ffdf80bcdaa admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:32.709135 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251f840-5a5c-492b-9ec3-8ffdf80bcdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:32.709407 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251f840-5a5c-492b-9ec3-8ffdf80bcdaa admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:32.710122 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251f840-5a5c-492b-9ec3-8ffdf80bcdaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:32.712032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251f840-5a5c-492b-9ec3-8ffdf80bcdaa admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:32.735115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 998/1971] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:32 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:32.737365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721c26f-150b-46c9-aa8b-cabcffffa4a0 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:32.748185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721c26f-150b-46c9-aa8b-cabcffffa4a0 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:32.752822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f86fb1-d01e-4735-808f-65d25e513a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:32.753480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f86fb1-d01e-4735-808f-65d25e513a02 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:32.753765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f86fb1-d01e-4735-808f-65d25e513a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:32.754083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f86fb1-d01e-4735-808f-65d25e513a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:32.754809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f86fb1-d01e-4735-808f-65d25e513a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:32.756397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f86fb1-d01e-4735-808f-65d25e513a02 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:32.766195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 974/1972] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:32.769872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251f840-5a5c-492b-9ec3-8ffdf80bcdaa admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:32.780344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251f840-5a5c-492b-9ec3-8ffdf80bcdaa admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:32.784800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad29e9-4180-4ccb-a0a6-4ad61883848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:32.785444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad29e9-4180-4ccb-a0a6-4ad61883848f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:32.785744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad29e9-4180-4ccb-a0a6-4ad61883848f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:32.786036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad29e9-4180-4ccb-a0a6-4ad61883848f admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:32.786757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad29e9-4180-4ccb-a0a6-4ad61883848f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:32.788330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad29e9-4180-4ccb-a0a6-4ad61883848f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:32.799295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 999/1973] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:32.803352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f86fb1-d01e-4735-808f-65d25e513a02 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:32.813368 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f86fb1-d01e-4735-808f-65d25e513a02 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:32.817969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bfe2a-e7c2-4e4c-993a-a2be9b172510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:32.818844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bfe2a-e7c2-4e4c-993a-a2be9b172510 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:32.819130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bfe2a-e7c2-4e4c-993a-a2be9b172510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:32.819410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bfe2a-e7c2-4e4c-993a-a2be9b172510 admin admin] PATH_INFO: `/v3/projects/2f5eb28a972a44cd8f807a32fc2143ee/users/067e0d4ed1474107821ff26865b5a787/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:32.833572 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bfe2a-e7c2-4e4c-993a-a2be9b172510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f5eb28a972a44cd8f807a32fc2143ee, user_id=067e0d4ed1474107821ff26865b5a787, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:32.836260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bfe2a-e7c2-4e4c-993a-a2be9b172510 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:32.850251 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 975/1974] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:32 2022] PUT /identity/v3/projects/2f5eb28a972a44cd8f807a32fc2143ee/users/067e0d4ed1474107821ff26865b5a787/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:32.862561 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb48961-81e1-405b-a0a7-ebe76a6e8a21 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:32.862849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb48961-81e1-405b-a0a7-ebe76a6e8a21 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:32.863146 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb48961-81e1-405b-a0a7-ebe76a6e8a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:32.890469 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0fb48961-81e1-405b-a0a7-ebe76a6e8a21 None None] MFA Rules not processed for user `067e0d4ed1474107821ff26865b5a787`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:32.891624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0fb48961-81e1-405b-a0a7-ebe76a6e8a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:32.930567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:32.958447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1000/1975] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:32.962892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bfe2a-e7c2-4e4c-993a-a2be9b172510 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:33.034075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bfe2a-e7c2-4e4c-993a-a2be9b172510 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:33.038692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcf91d-1247-4791-bcbf-f7e513133ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:33.039357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcf91d-1247-4791-bcbf-f7e513133ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.039639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcf91d-1247-4791-bcbf-f7e513133ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.039912 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcf91d-1247-4791-bcbf-f7e513133ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.041274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcf91d-1247-4791-bcbf-f7e513133ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:33.042675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcf91d-1247-4791-bcbf-f7e513133ec9 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:33.071287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 976/1976] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:32 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:33.075430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb48961-81e1-405b-a0a7-ebe76a6e8a21 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:33.086556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb48961-81e1-405b-a0a7-ebe76a6e8a21 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:33.145993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbf2d3-80c7-4a9f-8788-2fd04718ca14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:33.146578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbf2d3-80c7-4a9f-8788-2fd04718ca14 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.146863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbf2d3-80c7-4a9f-8788-2fd04718ca14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.147142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbf2d3-80c7-4a9f-8788-2fd04718ca14 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.147855 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbf2d3-80c7-4a9f-8788-2fd04718ca14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:33.149792 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbf2d3-80c7-4a9f-8788-2fd04718ca14 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:33.180193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1001/1977] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:33 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:33.184640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcf91d-1247-4791-bcbf-f7e513133ec9 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:33.195141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcf91d-1247-4791-bcbf-f7e513133ec9 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:33.200098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5c497-d094-42a2-b03d-5bee1122dee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:33.200953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5c497-d094-42a2-b03d-5bee1122dee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.201237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5c497-d094-42a2-b03d-5bee1122dee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.201540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5c497-d094-42a2-b03d-5bee1122dee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.202302 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5c497-d094-42a2-b03d-5bee1122dee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:33.203895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5c497-d094-42a2-b03d-5bee1122dee4 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:33.214450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 977/1978] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:33.218889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbf2d3-80c7-4a9f-8788-2fd04718ca14 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:33.229018 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbf2d3-80c7-4a9f-8788-2fd04718ca14 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:33.233555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c93a31-636b-4cf9-bf98-149d43f4be7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:33.234244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c93a31-636b-4cf9-bf98-149d43f4be7c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.234790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c93a31-636b-4cf9-bf98-149d43f4be7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.235063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c93a31-636b-4cf9-bf98-149d43f4be7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.235813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c93a31-636b-4cf9-bf98-149d43f4be7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:33.237468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c93a31-636b-4cf9-bf98-149d43f4be7c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:33.247387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1002/1979] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:33.251512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5c497-d094-42a2-b03d-5bee1122dee4 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:33.269201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5c497-d094-42a2-b03d-5bee1122dee4 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:33.269201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fea342-6c64-4344-a439-27697d2c9825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:33.269201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fea342-6c64-4344-a439-27697d2c9825 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.269201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fea342-6c64-4344-a439-27697d2c9825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.270273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fea342-6c64-4344-a439-27697d2c9825 admin admin] PATH_INFO: `/v3/projects/d4952b41dd134b2c9085b9c9e66642fd/users/c00233dc419240c7887a173c814487c6/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.284679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fea342-6c64-4344-a439-27697d2c9825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4952b41dd134b2c9085b9c9e66642fd, user_id=c00233dc419240c7887a173c814487c6, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:33.287410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fea342-6c64-4344-a439-27697d2c9825 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:33.300945 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 978/1980] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:33 2022] PUT /identity/v3/projects/d4952b41dd134b2c9085b9c9e66642fd/users/c00233dc419240c7887a173c814487c6/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:33.311471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c4c55-377b-4bac-bb52-4f5319315988 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.311765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c4c55-377b-4bac-bb52-4f5319315988 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.312074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c4c55-377b-4bac-bb52-4f5319315988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.340870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d25c4c55-377b-4bac-bb52-4f5319315988 None None] MFA Rules not processed for user `c00233dc419240c7887a173c814487c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:33.342860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d25c4c55-377b-4bac-bb52-4f5319315988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:33.408911 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1003/1981] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:33.413329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fea342-6c64-4344-a439-27697d2c9825 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:33.432291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:33.477155 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fea342-6c64-4344-a439-27697d2c9825 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:33.481825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93093008-a152-4abe-846f-65a26820ec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:33.482471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93093008-a152-4abe-846f-65a26820ec25 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.482752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93093008-a152-4abe-846f-65a26820ec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.483024 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93093008-a152-4abe-846f-65a26820ec25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.483733 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93093008-a152-4abe-846f-65a26820ec25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:33.485864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93093008-a152-4abe-846f-65a26820ec25 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:33.515833 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 979/1982] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:33 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:33.519841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c4c55-377b-4bac-bb52-4f5319315988 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:33.532401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c4c55-377b-4bac-bb52-4f5319315988 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:33.574368 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:33.588119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b8896-5858-452e-915a-57bfcc767fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:33.588819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b8896-5858-452e-915a-57bfcc767fb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.589109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b8896-5858-452e-915a-57bfcc767fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.589384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b8896-5858-452e-915a-57bfcc767fb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.590139 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b8896-5858-452e-915a-57bfcc767fb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:33.592103 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b8896-5858-452e-915a-57bfcc767fb2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:33.619016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1004/1983] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:33 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:33.620102 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93093008-a152-4abe-846f-65a26820ec25 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:33.632083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93093008-a152-4abe-846f-65a26820ec25 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:33.636748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028c49a-380b-4b71-8571-ad07424bfd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:33.637412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028c49a-380b-4b71-8571-ad07424bfd13 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.637695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028c49a-380b-4b71-8571-ad07424bfd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.640433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028c49a-380b-4b71-8571-ad07424bfd13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.641225 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8028c49a-380b-4b71-8571-ad07424bfd13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:33.642802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8028c49a-380b-4b71-8571-ad07424bfd13 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:33.655099 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 980/1984] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:33.660193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b8896-5858-452e-915a-57bfcc767fb2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:33.670735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b8896-5858-452e-915a-57bfcc767fb2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:33.677305 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ce167-b6f4-407a-8cc4-ea4a85c01f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:33.678259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ce167-b6f4-407a-8cc4-ea4a85c01f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.678550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ce167-b6f4-407a-8cc4-ea4a85c01f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.678826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ce167-b6f4-407a-8cc4-ea4a85c01f43 admin admin] PATH_INFO: `/v3/projects/694fb73f629a4f729c7b44f8d4e96fec/users/1212bf775b874408b98ffe613809d6cb/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.696440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ce167-b6f4-407a-8cc4-ea4a85c01f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694fb73f629a4f729c7b44f8d4e96fec, user_id=1212bf775b874408b98ffe613809d6cb, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:33.699163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ce167-b6f4-407a-8cc4-ea4a85c01f43 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:33.717766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1005/1985] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:33 2022] PUT /identity/v3/projects/694fb73f629a4f729c7b44f8d4e96fec/users/1212bf775b874408b98ffe613809d6cb/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:33.723646 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028c49a-380b-4b71-8571-ad07424bfd13 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:33.743435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028c49a-380b-4b71-8571-ad07424bfd13 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:33.842551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcffa26-bf32-4362-bf6a-523d7d14db00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:33.843303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcffa26-bf32-4362-bf6a-523d7d14db00 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.843599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcffa26-bf32-4362-bf6a-523d7d14db00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.843878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcffa26-bf32-4362-bf6a-523d7d14db00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.844620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcffa26-bf32-4362-bf6a-523d7d14db00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:33.846248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcffa26-bf32-4362-bf6a-523d7d14db00 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:33.862978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 981/1986] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:33.864470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ce167-b6f4-407a-8cc4-ea4a85c01f43 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:33.875999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ce167-b6f4-407a-8cc4-ea4a85c01f43 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:33.881013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b64d4-7195-4760-8736-12686b8cc425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:33.881926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b64d4-7195-4760-8736-12686b8cc425 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.882346 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b64d4-7195-4760-8736-12686b8cc425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.882630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b64d4-7195-4760-8736-12686b8cc425 admin admin] PATH_INFO: `/v3/projects/694fb73f629a4f729c7b44f8d4e96fec/users/1212bf775b874408b98ffe613809d6cb/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.884630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b64d4-7195-4760-8736-12686b8cc425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694fb73f629a4f729c7b44f8d4e96fec, user_id=1212bf775b874408b98ffe613809d6cb, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:33.887407 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b64d4-7195-4760-8736-12686b8cc425 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:33.899660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1006/1987] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:33 2022] PUT /identity/v3/projects/694fb73f629a4f729c7b44f8d4e96fec/users/1212bf775b874408b98ffe613809d6cb/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:33.910598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e38cca-5be2-4bd0-9826-cdf7737d71b8 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:33.910915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e38cca-5be2-4bd0-9826-cdf7737d71b8 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:33.911196 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e38cca-5be2-4bd0-9826-cdf7737d71b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:33.950983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b4e38cca-5be2-4bd0-9826-cdf7737d71b8 None None] MFA Rules not processed for user `1212bf775b874408b98ffe613809d6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:33.952176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b4e38cca-5be2-4bd0-9826-cdf7737d71b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:34.022654 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 982/1988] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:34.033366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b64d4-7195-4760-8736-12686b8cc425 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:34.035092 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-299b64d4-7195-4760-8736-12686b8cc425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:34.091171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b64d4-7195-4760-8736-12686b8cc425 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:34.094835 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a267afa-fad0-438d-b554-aeb179a3f455 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:34.095707 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a267afa-fad0-438d-b554-aeb179a3f455 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:34.096000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a267afa-fad0-438d-b554-aeb179a3f455 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:34.096278 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a267afa-fad0-438d-b554-aeb179a3f455 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:34.105198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a267afa-fad0-438d-b554-aeb179a3f455 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:34.107193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a267afa-fad0-438d-b554-aeb179a3f455 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:34.117284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1007/1989] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:34.680102 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e38cca-5be2-4bd0-9826-cdf7737d71b8 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:34.689575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e38cca-5be2-4bd0-9826-cdf7737d71b8 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:34.693149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090e4fd-f464-45ac-a660-35de6dae924e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:34.693790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090e4fd-f464-45ac-a660-35de6dae924e service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:34.694105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090e4fd-f464-45ac-a660-35de6dae924e service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:34.694405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090e4fd-f464-45ac-a660-35de6dae924e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:34.704496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:34.756262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090e4fd-f464-45ac-a660-35de6dae924e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:34.757774 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090e4fd-f464-45ac-a660-35de6dae924e service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:34.767196 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 983/1990] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:36.457329 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f33ca-b8bb-42f7-a4fb-cb477d63f9ff None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:36.457865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f33ca-b8bb-42f7-a4fb-cb477d63f9ff None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:36.457865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f33ca-b8bb-42f7-a4fb-cb477d63f9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:36.477803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5e0f33ca-b8bb-42f7-a4fb-cb477d63f9ff None None] MFA Rules not processed for user `0f4376954dbd4402ae5a990f1a56baf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:36.479447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5e0f33ca-b8bb-42f7-a4fb-cb477d63f9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:36.535632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1008/1991] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:44.316376 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090e4fd-f464-45ac-a660-35de6dae924e service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:44.317826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6090e4fd-f464-45ac-a660-35de6dae924e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:44.328959 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090e4fd-f464-45ac-a660-35de6dae924e service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:44.332880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41eb34-675d-41d3-88aa-0279f0fd7bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:44.333608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41eb34-675d-41d3-88aa-0279f0fd7bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:44.333994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41eb34-675d-41d3-88aa-0279f0fd7bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:44.334338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41eb34-675d-41d3-88aa-0279f0fd7bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:44.402165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41eb34-675d-41d3-88aa-0279f0fd7bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:44.404206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41eb34-675d-41d3-88aa-0279f0fd7bf8 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:44.415419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 984/1992] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:46.528709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:46.821273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f33ca-b8bb-42f7-a4fb-cb477d63f9ff None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:46.894712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f33ca-b8bb-42f7-a4fb-cb477d63f9ff None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:46.899288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:46.900161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:46.900442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:46.900714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] PATH_INFO: `/v3/users/067e0d4ed1474107821ff26865b5a787` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:46.901978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] RBAC: Authorizing `identity:delete_user(user_id=067e0d4ed1474107821ff26865b5a787)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:46.903909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:46.958428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '067e0d4ed1474107821ff26865b5a787'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:46.967048 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '067e0d4ed1474107821ff26865b5a787'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:46.967739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '067e0d4ed1474107821ff26865b5a787'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:46.975196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '067e0d4ed1474107821ff26865b5a787'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:46.975904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '067e0d4ed1474107821ff26865b5a787'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:46.995536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1009/1993] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:46 2022] DELETE /identity/v3/users/067e0d4ed1474107821ff26865b5a787 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:47.000638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f5fe6-d761-4ff7-b863-8fd0fccd6148 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:47.001097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f5fe6-d761-4ff7-b863-8fd0fccd6148 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:47.001185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f5fe6-d761-4ff7-b863-8fd0fccd6148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:47.024183 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-927f5fe6-d761-4ff7-b863-8fd0fccd6148 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:47.025406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-927f5fe6-d761-4ff7-b863-8fd0fccd6148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:47.089698 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 985/1994] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:47.100197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:47.102606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:47.184654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd61d8-3da9-4a28-bdad-42b4ac9a601f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:47.194717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d21b02-f9d5-4e13-8c6d-8dc0674388f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:47.195408 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d21b02-f9d5-4e13-8c6d-8dc0674388f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:47.195688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d21b02-f9d5-4e13-8c6d-8dc0674388f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:47.195960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d21b02-f9d5-4e13-8c6d-8dc0674388f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:47.205979 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d21b02-f9d5-4e13-8c6d-8dc0674388f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:47.207428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d21b02-f9d5-4e13-8c6d-8dc0674388f1 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:47.218390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1010/1995] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:47.314706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f5fe6-d761-4ff7-b863-8fd0fccd6148 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:47.316285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-927f5fe6-d761-4ff7-b863-8fd0fccd6148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:47.332409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f5fe6-d761-4ff7-b863-8fd0fccd6148 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:47.338600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:47.339740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:47.341397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:47.341831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] PATH_INFO: `/v3/users/8a34d144c07b4d47b78ef8249aeaf46a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:47.343334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a34d144c07b4d47b78ef8249aeaf46a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:47.347753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:47.416594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:47.420077 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a34d144c07b4d47b78ef8249aeaf46a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:47.421396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a34d144c07b4d47b78ef8249aeaf46a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:47.422841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a34d144c07b4d47b78ef8249aeaf46a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:47.448216 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a34d144c07b4d47b78ef8249aeaf46a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:47.463967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a34d144c07b4d47b78ef8249aeaf46a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:47.507442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 986/1996] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:47 2022] DELETE /identity/v3/users/8a34d144c07b4d47b78ef8249aeaf46a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:47.511634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23681890-e1e9-4124-8886-e2445df4b871 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:47.511920 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23681890-e1e9-4124-8886-e2445df4b871 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:47.512183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23681890-e1e9-4124-8886-e2445df4b871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:47.538315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-23681890-e1e9-4124-8886-e2445df4b871 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:47.540229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-23681890-e1e9-4124-8886-e2445df4b871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:47.572215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:47.634361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1011/1997] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:47.651976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:47.654631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:47.730095 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932e276-901f-4c64-8aa6-93d3b8dd74ae admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:47.735532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b40a18-1545-4b7b-a8d8-05525d3ea99a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:47.736319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b40a18-1545-4b7b-a8d8-05525d3ea99a service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:47.736848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b40a18-1545-4b7b-a8d8-05525d3ea99a service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:47.737379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b40a18-1545-4b7b-a8d8-05525d3ea99a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:47.752100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b40a18-1545-4b7b-a8d8-05525d3ea99a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:47.753949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b40a18-1545-4b7b-a8d8-05525d3ea99a service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:47.774540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 987/1998] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:47.872520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23681890-e1e9-4124-8886-e2445df4b871 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:47.874358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-23681890-e1e9-4124-8886-e2445df4b871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:47.891844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23681890-e1e9-4124-8886-e2445df4b871 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:47.897320 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:47.898767 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:47.899768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:47.900151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] PATH_INFO: `/v3/projects/2f5eb28a972a44cd8f807a32fc2143ee` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:47.905124 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f5eb28a972a44cd8f807a32fc2143ee)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:47.908049 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:47.984688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f5eb28a972a44cd8f807a32fc2143ee', 'request_id': 'req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:47.998563 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f5eb28a972a44cd8f807a32fc2143ee', 'request_id': 'req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:48.007293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f5eb28a972a44cd8f807a32fc2143ee', 'request_id': 'req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:48.029242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] The token cache is being invalidate because project 2f5eb28a972a44cd8f807a32fc2143ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:48.029691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:48.030842 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:48.033301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1012/1999] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:47 2022] DELETE /identity/v3/projects/2f5eb28a972a44cd8f807a32fc2143ee => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:48.037880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b40a18-1545-4b7b-a8d8-05525d3ea99a service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:48.055301 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b40a18-1545-4b7b-a8d8-05525d3ea99a service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:48.156751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:48.157698 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:48.158146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:48.158541 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] PATH_INFO: `/v3/users/c00233dc419240c7887a173c814487c6` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:48.160791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c00233dc419240c7887a173c814487c6)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:48.163405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:48.236930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c00233dc419240c7887a173c814487c6'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:48.238344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c00233dc419240c7887a173c814487c6'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:48.239634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c00233dc419240c7887a173c814487c6'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:48.252949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c00233dc419240c7887a173c814487c6'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:48.261550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c00233dc419240c7887a173c814487c6'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:48.267418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:48.289872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 988/2000] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:48 2022] DELETE /identity/v3/users/c00233dc419240c7887a173c814487c6 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:48.768458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:48.770138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:48.830743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2105b47-9a0f-4b9c-9eed-61b74754dfa9 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:48.952168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:48.953059 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:48.953345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:48.953625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] PATH_INFO: `/v3/projects/3f20ce7126484948b2ebd2c0a4985ec6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:48.954970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f20ce7126484948b2ebd2c0a4985ec6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:48.956940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:49.020217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f20ce7126484948b2ebd2c0a4985ec6', 'request_id': 'req-72d92247-3d79-4354-8f17-045e61123a11'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:49.024830 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:49.037665 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f20ce7126484948b2ebd2c0a4985ec6', 'request_id': 'req-72d92247-3d79-4354-8f17-045e61123a11'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:49.046487 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f20ce7126484948b2ebd2c0a4985ec6', 'request_id': 'req-72d92247-3d79-4354-8f17-045e61123a11'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:49.058711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] The token cache is being invalidate because project 3f20ce7126484948b2ebd2c0a4985ec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:49.059326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:49.060017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:49.061870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1013/2001] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:48 2022] DELETE /identity/v3/projects/3f20ce7126484948b2ebd2c0a4985ec6 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:49.065759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:49.070366 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:49.084324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab291962-2e3a-467e-b60b-f6f1564504b8 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:49.152039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:49.156596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:49.156596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:49.156596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] PATH_INFO: `/v3/users/6fe5a0a5e23d4778bf35ad04dbb7cf6b` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:49.156596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe5a0a5e23d4778bf35ad04dbb7cf6b)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:49.157939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:49.235368 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe5a0a5e23d4778bf35ad04dbb7cf6b'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:49.236304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe5a0a5e23d4778bf35ad04dbb7cf6b'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:49.237123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe5a0a5e23d4778bf35ad04dbb7cf6b'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:49.244947 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe5a0a5e23d4778bf35ad04dbb7cf6b'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:49.254657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe5a0a5e23d4778bf35ad04dbb7cf6b'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:49.276998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 989/2002] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:49 2022] DELETE /identity/v3/users/6fe5a0a5e23d4778bf35ad04dbb7cf6b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:49.669831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:49.671327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:49.696746 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d92247-3d79-4354-8f17-045e61123a11 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:49.842354 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:49.843214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:49.843575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:49.844826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] PATH_INFO: `/v3/projects/d4952b41dd134b2c9085b9c9e66642fd` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:49.845714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4952b41dd134b2c9085b9c9e66642fd)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:49.862434 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:49.961179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4952b41dd134b2c9085b9c9e66642fd', 'request_id': 'req-e2687f31-f361-4806-9611-6c0441073e7e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:49.975196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4952b41dd134b2c9085b9c9e66642fd', 'request_id': 'req-e2687f31-f361-4806-9611-6c0441073e7e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:49.994266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4952b41dd134b2c9085b9c9e66642fd', 'request_id': 'req-e2687f31-f361-4806-9611-6c0441073e7e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.004600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] The token cache is being invalidate because project d4952b41dd134b2c9085b9c9e66642fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:50.004907 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.005744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.008310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1014/2003] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:49 2022] DELETE /identity/v3/projects/d4952b41dd134b2c9085b9c9e66642fd => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:50.012377 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:50.014990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:50.045741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011d8ef-2b26-4c6e-aa33-3a9d99a9df9d admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:50.142564 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6395cf-b060-4074-937c-7a3866ef8707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:50.144713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6395cf-b060-4074-937c-7a3866ef8707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:50.145231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6395cf-b060-4074-937c-7a3866ef8707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:50.145868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6395cf-b060-4074-937c-7a3866ef8707 admin admin] PATH_INFO: `/v3/users/1212bf775b874408b98ffe613809d6cb` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:50.149027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6395cf-b060-4074-937c-7a3866ef8707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1212bf775b874408b98ffe613809d6cb)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:50.156706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6395cf-b060-4074-937c-7a3866ef8707 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:50.169532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:50.290616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2a6395cf-b060-4074-937c-7a3866ef8707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1212bf775b874408b98ffe613809d6cb'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.292306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2a6395cf-b060-4074-937c-7a3866ef8707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1212bf775b874408b98ffe613809d6cb'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.294415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2a6395cf-b060-4074-937c-7a3866ef8707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1212bf775b874408b98ffe613809d6cb'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.308891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2a6395cf-b060-4074-937c-7a3866ef8707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1212bf775b874408b98ffe613809d6cb'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.338335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2a6395cf-b060-4074-937c-7a3866ef8707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1212bf775b874408b98ffe613809d6cb'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.344197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:50.345950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:50.384328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2687f31-f361-4806-9611-6c0441073e7e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:50.393332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 990/2004] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:50 2022] DELETE /identity/v3/users/1212bf775b874408b98ffe613809d6cb => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:50.395032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cdec41-ad1f-445d-8655-24426d97adef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:50.396319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdec41-ad1f-445d-8655-24426d97adef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:50.396698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdec41-ad1f-445d-8655-24426d97adef admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:50.397443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdec41-ad1f-445d-8655-24426d97adef admin admin] PATH_INFO: `/v3/projects/68ae482fad824961852e91f21e7c13a2` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:50.398604 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cdec41-ad1f-445d-8655-24426d97adef admin admin] RBAC: Authorizing `identity:delete_project(project_id=68ae482fad824961852e91f21e7c13a2)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:50.400803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cdec41-ad1f-445d-8655-24426d97adef admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:50.527005 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e4cdec41-ad1f-445d-8655-24426d97adef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68ae482fad824961852e91f21e7c13a2', 'request_id': 'req-e4cdec41-ad1f-445d-8655-24426d97adef'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.553651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e4cdec41-ad1f-445d-8655-24426d97adef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68ae482fad824961852e91f21e7c13a2', 'request_id': 'req-e4cdec41-ad1f-445d-8655-24426d97adef'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.571400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e4cdec41-ad1f-445d-8655-24426d97adef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68ae482fad824961852e91f21e7c13a2', 'request_id': 'req-e4cdec41-ad1f-445d-8655-24426d97adef'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.573774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:50.597583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e4cdec41-ad1f-445d-8655-24426d97adef admin admin] The token cache is being invalidate because project 68ae482fad824961852e91f21e7c13a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:50.598221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e4cdec41-ad1f-445d-8655-24426d97adef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.599032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e4cdec41-ad1f-445d-8655-24426d97adef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.601689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1015/2005] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:50 2022] DELETE /identity/v3/projects/68ae482fad824961852e91f21e7c13a2 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:50.612644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aea158-d4a2-4910-a66a-a80fb5f83071 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:50.612785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aea158-d4a2-4910-a66a-a80fb5f83071 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:50.613240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aea158-d4a2-4910-a66a-a80fb5f83071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:50.657211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-12aea158-d4a2-4910-a66a-a80fb5f83071 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:50.658818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-12aea158-d4a2-4910-a66a-a80fb5f83071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:50.776646 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 991/2006] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:50.779487 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359ffa3-7be4-45d3-83c2-16eeb6e4ea9c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:50.779851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359ffa3-7be4-45d3-83c2-16eeb6e4ea9c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:50.780638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359ffa3-7be4-45d3-83c2-16eeb6e4ea9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:50.804290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7359ffa3-7be4-45d3-83c2-16eeb6e4ea9c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:50.806036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7359ffa3-7be4-45d3-83c2-16eeb6e4ea9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:50.818033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1016/2007] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:50.819215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aea158-d4a2-4910-a66a-a80fb5f83071 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:50.830119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aea158-d4a2-4910-a66a-a80fb5f83071 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:50.834986 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64547bff-aa74-4fcd-a261-c860a09cc6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:50.835777 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64547bff-aa74-4fcd-a261-c860a09cc6dc admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:50.836096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64547bff-aa74-4fcd-a261-c860a09cc6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:50.836403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64547bff-aa74-4fcd-a261-c860a09cc6dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:50.837137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64547bff-aa74-4fcd-a261-c860a09cc6dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:50.837450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64547bff-aa74-4fcd-a261-c860a09cc6dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:50.839208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64547bff-aa74-4fcd-a261-c860a09cc6dc admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:50.853987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 992/2008] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:40:50.858493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64547bff-aa74-4fcd-a261-c860a09cc6dc admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:50.858660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359ffa3-7be4-45d3-83c2-16eeb6e4ea9c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:50.860819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7359ffa3-7be4-45d3-83c2-16eeb6e4ea9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:50.874990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359ffa3-7be4-45d3-83c2-16eeb6e4ea9c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:50.879722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64547bff-aa74-4fcd-a261-c860a09cc6dc admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:50.879988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe76034-6f1e-4af5-bf41-f0d78cf03f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:50.880863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe76034-6f1e-4af5-bf41-f0d78cf03f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:50.881148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe76034-6f1e-4af5-bf41-f0d78cf03f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:50.881423 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe76034-6f1e-4af5-bf41-f0d78cf03f62 admin admin] PATH_INFO: `/v3/projects/694fb73f629a4f729c7b44f8d4e96fec` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:50.882628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe76034-6f1e-4af5-bf41-f0d78cf03f62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=694fb73f629a4f729c7b44f8d4e96fec)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:50.884748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe76034-6f1e-4af5-bf41-f0d78cf03f62 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:50.886504 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0923e33-c57b-4d9c-8326-0bc9263ede57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:50.887254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0923e33-c57b-4d9c-8326-0bc9263ede57 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:50.887639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0923e33-c57b-4d9c-8326-0bc9263ede57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:50.887985 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0923e33-c57b-4d9c-8326-0bc9263ede57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:50.888757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0923e33-c57b-4d9c-8326-0bc9263ede57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:50.898332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0923e33-c57b-4d9c-8326-0bc9263ede57 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:50.934141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 993/2009] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:50 2022] POST /identity/v3/projects => generated 387 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:50.938133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0923e33-c57b-4d9c-8326-0bc9263ede57 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:50.942409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-afe76034-6f1e-4af5-bf41-f0d78cf03f62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '694fb73f629a4f729c7b44f8d4e96fec', 'request_id': 'req-afe76034-6f1e-4af5-bf41-f0d78cf03f62'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.952117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-afe76034-6f1e-4af5-bf41-f0d78cf03f62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '694fb73f629a4f729c7b44f8d4e96fec', 'request_id': 'req-afe76034-6f1e-4af5-bf41-f0d78cf03f62'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.959982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-afe76034-6f1e-4af5-bf41-f0d78cf03f62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '694fb73f629a4f729c7b44f8d4e96fec', 'request_id': 'req-afe76034-6f1e-4af5-bf41-f0d78cf03f62'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.972289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-afe76034-6f1e-4af5-bf41-f0d78cf03f62 admin admin] The token cache is being invalidate because project 694fb73f629a4f729c7b44f8d4e96fec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:50.972592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-afe76034-6f1e-4af5-bf41-f0d78cf03f62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.973257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-afe76034-6f1e-4af5-bf41-f0d78cf03f62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:50.975007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1017/2010] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:50 2022] DELETE /identity/v3/projects/694fb73f629a4f729c7b44f8d4e96fec => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:50.980660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca80521-234b-46fb-a1d9-cec94d6cb2f1 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:50.980951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca80521-234b-46fb-a1d9-cec94d6cb2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:50.981798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca80521-234b-46fb-a1d9-cec94d6cb2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:50.998031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:51.004002 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2ca80521-234b-46fb-a1d9-cec94d6cb2f1 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:51.005300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2ca80521-234b-46fb-a1d9-cec94d6cb2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:51.021345 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0923e33-c57b-4d9c-8326-0bc9263ede57 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.029467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa56660-aec6-43a6-b9f5-3b7a60189ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.030283 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa56660-aec6-43a6-b9f5-3b7a60189ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.030631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa56660-aec6-43a6-b9f5-3b7a60189ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.030970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa56660-aec6-43a6-b9f5-3b7a60189ed0 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.031757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa56660-aec6-43a6-b9f5-3b7a60189ed0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.033796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa56660-aec6-43a6-b9f5-3b7a60189ed0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.071387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 994/2011] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:50 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:51.072842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa56660-aec6-43a6-b9f5-3b7a60189ed0 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.075894 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7aa56660-aec6-43a6-b9f5-3b7a60189ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:51.085646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1018/2012] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:51.099882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa56660-aec6-43a6-b9f5-3b7a60189ed0 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.107966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769df9d-0838-47fa-8c95-c940aed1f676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.108845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769df9d-0838-47fa-8c95-c940aed1f676 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.109214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769df9d-0838-47fa-8c95-c940aed1f676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.109846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769df9d-0838-47fa-8c95-c940aed1f676 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.110257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db6cc3-9574-47e1-ac7b-f591cb04a653 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.110590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db6cc3-9574-47e1-ac7b-f591cb04a653 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.110884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db6cc3-9574-47e1-ac7b-f591cb04a653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.111467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769df9d-0838-47fa-8c95-c940aed1f676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.113858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769df9d-0838-47fa-8c95-c940aed1f676 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.125889 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 995/2013] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:51.130734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769df9d-0838-47fa-8c95-c940aed1f676 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.133546 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-27db6cc3-9574-47e1-ac7b-f591cb04a653 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:51.134988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-27db6cc3-9574-47e1-ac7b-f591cb04a653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:51.147624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1019/2014] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:51.150260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769df9d-0838-47fa-8c95-c940aed1f676 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.154770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db6cc3-9574-47e1-ac7b-f591cb04a653 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.159934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7dca88-1ea6-4283-944a-366642bb3e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.160852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dca88-1ea6-4283-944a-366642bb3e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.161210 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dca88-1ea6-4283-944a-366642bb3e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.161560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dca88-1ea6-4283-944a-366642bb3e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.162416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7dca88-1ea6-4283-944a-366642bb3e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.164250 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7dca88-1ea6-4283-944a-366642bb3e35 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.167241 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db6cc3-9574-47e1-ac7b-f591cb04a653 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.175873 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6448111-81b3-4bb5-9484-573601a77ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.177506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6448111-81b3-4bb5-9484-573601a77ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.178138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6448111-81b3-4bb5-9484-573601a77ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.178751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6448111-81b3-4bb5-9484-573601a77ccc admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.180011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6448111-81b3-4bb5-9484-573601a77ccc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:51.180678 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6448111-81b3-4bb5-9484-573601a77ccc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.181849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 996/2015] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:51.183071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6448111-81b3-4bb5-9484-573601a77ccc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.185883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7dca88-1ea6-4283-944a-366642bb3e35 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.204162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7dca88-1ea6-4283-944a-366642bb3e35 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.208728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1020/2016] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:40:51.209862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50028294-fdd3-4c6d-9c57-d7cc33618e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.210045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6448111-81b3-4bb5-9484-573601a77ccc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.211596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50028294-fdd3-4c6d-9c57-d7cc33618e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.211888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50028294-fdd3-4c6d-9c57-d7cc33618e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.212240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50028294-fdd3-4c6d-9c57-d7cc33618e4c admin admin] PATH_INFO: `/v3/projects/7b23aca8b9e546e7af0ef5c2e150b8fe/users/25750a71e10a4fb9ba9c80bb87179e26/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.228517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6448111-81b3-4bb5-9484-573601a77ccc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.232335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50028294-fdd3-4c6d-9c57-d7cc33618e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b23aca8b9e546e7af0ef5c2e150b8fe, user_id=25750a71e10a4fb9ba9c80bb87179e26, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.233526 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d96442-97fe-4b3b-8c95-b84c1c0b19ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.234246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d96442-97fe-4b3b-8c95-b84c1c0b19ad admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.234921 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d96442-97fe-4b3b-8c95-b84c1c0b19ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.235202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d96442-97fe-4b3b-8c95-b84c1c0b19ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.235294 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50028294-fdd3-4c6d-9c57-d7cc33618e4c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.235897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d96442-97fe-4b3b-8c95-b84c1c0b19ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.238165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d96442-97fe-4b3b-8c95-b84c1c0b19ad admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.264298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 997/2017] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:51 2022] PUT /identity/v3/projects/7b23aca8b9e546e7af0ef5c2e150b8fe/users/25750a71e10a4fb9ba9c80bb87179e26/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:51.268856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf2913-1a13-4f39-bbef-ebdc13c12738 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.269278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf2913-1a13-4f39-bbef-ebdc13c12738 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.269734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf2913-1a13-4f39-bbef-ebdc13c12738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.287064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1021/2018] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:51 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:51.292263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d96442-97fe-4b3b-8c95-b84c1c0b19ad admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.320608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-5adf2913-1a13-4f39-bbef-ebdc13c12738 None None] MFA Rules not processed for user `25750a71e10a4fb9ba9c80bb87179e26`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:51.322173 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5adf2913-1a13-4f39-bbef-ebdc13c12738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:51.337161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:51.395749 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d96442-97fe-4b3b-8c95-b84c1c0b19ad admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.400655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eee35-f87e-430d-801c-690e0fca7bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.401475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eee35-f87e-430d-801c-690e0fca7bad admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.401766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eee35-f87e-430d-801c-690e0fca7bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.402076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eee35-f87e-430d-801c-690e0fca7bad admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.402781 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57eee35-f87e-430d-801c-690e0fca7bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.405302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57eee35-f87e-430d-801c-690e0fca7bad admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.413773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 998/2019] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:51.415016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf2913-1a13-4f39-bbef-ebdc13c12738 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.430800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf2913-1a13-4f39-bbef-ebdc13c12738 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.436597 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee711c-4a6e-4e93-bb76-5acc1a6f9c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.437358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee711c-4a6e-4e93-bb76-5acc1a6f9c4c admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.437637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee711c-4a6e-4e93-bb76-5acc1a6f9c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.437928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee711c-4a6e-4e93-bb76-5acc1a6f9c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.438755 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee711c-4a6e-4e93-bb76-5acc1a6f9c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.442850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1022/2020] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:51 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:51.442991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee711c-4a6e-4e93-bb76-5acc1a6f9c4c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.444347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eee35-f87e-430d-801c-690e0fca7bad admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.459321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eee35-f87e-430d-801c-690e0fca7bad admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.464917 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a42fd-0584-4f95-9308-0fd777d99ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.466363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a42fd-0584-4f95-9308-0fd777d99ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.466756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a42fd-0584-4f95-9308-0fd777d99ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.467285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a42fd-0584-4f95-9308-0fd777d99ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.468254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a42fd-0584-4f95-9308-0fd777d99ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.470687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a42fd-0584-4f95-9308-0fd777d99ac0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.483227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 999/2021] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:51 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:51.487575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee711c-4a6e-4e93-bb76-5acc1a6f9c4c admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.497629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1023/2022] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:51.501525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a42fd-0584-4f95-9308-0fd777d99ac0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.515910 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a42fd-0584-4f95-9308-0fd777d99ac0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.573951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee711c-4a6e-4e93-bb76-5acc1a6f9c4c admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.579338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864d9f5-92e1-4dff-8ac9-75827d95eb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.580051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864d9f5-92e1-4dff-8ac9-75827d95eb4e admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.580349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864d9f5-92e1-4dff-8ac9-75827d95eb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.580609 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864d9f5-92e1-4dff-8ac9-75827d95eb4e admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.581308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864d9f5-92e1-4dff-8ac9-75827d95eb4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.583341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864d9f5-92e1-4dff-8ac9-75827d95eb4e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.590799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62bdd5-57c0-4db3-b787-4598e6530e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.591474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62bdd5-57c0-4db3-b787-4598e6530e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.591779 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62bdd5-57c0-4db3-b787-4598e6530e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.592051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62bdd5-57c0-4db3-b787-4598e6530e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.592798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62bdd5-57c0-4db3-b787-4598e6530e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.593875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:51.594404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62bdd5-57c0-4db3-b787-4598e6530e3a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.609084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1024/2023] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:51.613149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62bdd5-57c0-4db3-b787-4598e6530e3a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.613597 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1000/2024] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:51 2022] POST /identity/v3/users => generated 300 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:51.617273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864d9f5-92e1-4dff-8ac9-75827d95eb4e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.624632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62bdd5-57c0-4db3-b787-4598e6530e3a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.627448 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864d9f5-92e1-4dff-8ac9-75827d95eb4e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.629695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b37bd-ba4e-4ce8-9345-806d1a46cecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.630605 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b37bd-ba4e-4ce8-9345-806d1a46cecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.630885 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b37bd-ba4e-4ce8-9345-806d1a46cecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.631157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b37bd-ba4e-4ce8-9345-806d1a46cecd admin admin] PATH_INFO: `/v3/projects/c5a44685abb34cf7833a66ae52330ae9/users/f7fefc97e5b441059c7916a9e7f838d7/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.632725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea40f8a-1aa9-42ef-9ce1-ad927da1cf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.633492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea40f8a-1aa9-42ef-9ce1-ad927da1cf74 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.633831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea40f8a-1aa9-42ef-9ce1-ad927da1cf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.634201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea40f8a-1aa9-42ef-9ce1-ad927da1cf74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.634935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea40f8a-1aa9-42ef-9ce1-ad927da1cf74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.636737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea40f8a-1aa9-42ef-9ce1-ad927da1cf74 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.648890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b37bd-ba4e-4ce8-9345-806d1a46cecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5a44685abb34cf7833a66ae52330ae9, user_id=f7fefc97e5b441059c7916a9e7f838d7, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.650229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1001/2025] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:51.651769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b37bd-ba4e-4ce8-9345-806d1a46cecd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.654186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea40f8a-1aa9-42ef-9ce1-ad927da1cf74 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.664679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea40f8a-1aa9-42ef-9ce1-ad927da1cf74 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.667940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1025/2026] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:51 2022] PUT /identity/v3/projects/c5a44685abb34cf7833a66ae52330ae9/users/f7fefc97e5b441059c7916a9e7f838d7/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:51.679319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30a6d4-6f0f-4542-8436-d453963b4d20 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.679602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30a6d4-6f0f-4542-8436-d453963b4d20 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.679877 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30a6d4-6f0f-4542-8436-d453963b4d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.711193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-bf30a6d4-6f0f-4542-8436-d453963b4d20 None None] MFA Rules not processed for user `f7fefc97e5b441059c7916a9e7f838d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:51.712413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bf30a6d4-6f0f-4542-8436-d453963b4d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:51.733868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6481a-1f25-4953-913a-34fecd061343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.735208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6481a-1f25-4953-913a-34fecd061343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.735534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6481a-1f25-4953-913a-34fecd061343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.735744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6481a-1f25-4953-913a-34fecd061343 admin admin] PATH_INFO: `/v3/projects/227fd9f301d54b7196d00e9577632b52/users/b7698cbabf9e4721bdd9c16d1c0b5d58/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.753657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6481a-1f25-4953-913a-34fecd061343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=227fd9f301d54b7196d00e9577632b52, user_id=b7698cbabf9e4721bdd9c16d1c0b5d58, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.757961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6481a-1f25-4953-913a-34fecd061343 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.777838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1002/2027] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:51 2022] PUT /identity/v3/projects/227fd9f301d54b7196d00e9577632b52/users/b7698cbabf9e4721bdd9c16d1c0b5d58/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:51.781741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6481a-1f25-4953-913a-34fecd061343 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.799831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1026/2028] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:51.803917 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30a6d4-6f0f-4542-8436-d453963b4d20 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.811750 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:51.816932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30a6d4-6f0f-4542-8436-d453963b4d20 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.856423 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6481a-1f25-4953-913a-34fecd061343 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.862026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64e198-d283-4cf0-beef-f095f28298ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.862699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64e198-d283-4cf0-beef-f095f28298ba admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.862982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64e198-d283-4cf0-beef-f095f28298ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.863258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64e198-d283-4cf0-beef-f095f28298ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.863985 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64e198-d283-4cf0-beef-f095f28298ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.865968 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64e198-d283-4cf0-beef-f095f28298ba admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.877801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1003/2029] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:51.885090 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64e198-d283-4cf0-beef-f095f28298ba admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.898613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d221a-ce30-47e9-b4b7-7742f3553f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.899293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d221a-ce30-47e9-b4b7-7742f3553f1b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.899577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d221a-ce30-47e9-b4b7-7742f3553f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.899852 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d221a-ce30-47e9-b4b7-7742f3553f1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.900394 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64e198-d283-4cf0-beef-f095f28298ba admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:51.900777 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d221a-ce30-47e9-b4b7-7742f3553f1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.902811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d221a-ce30-47e9-b4b7-7742f3553f1b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.906003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4affc-c4f5-4702-9ee7-c8a9ad97f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:51.907304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4affc-c4f5-4702-9ee7-c8a9ad97f8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.907589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4affc-c4f5-4702-9ee7-c8a9ad97f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.907862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4affc-c4f5-4702-9ee7-c8a9ad97f8a5 admin admin] PATH_INFO: `/v3/projects/227fd9f301d54b7196d00e9577632b52/users/b7698cbabf9e4721bdd9c16d1c0b5d58/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.910286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4affc-c4f5-4702-9ee7-c8a9ad97f8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=227fd9f301d54b7196d00e9577632b52, user_id=b7698cbabf9e4721bdd9c16d1c0b5d58, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:51.913997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4affc-c4f5-4702-9ee7-c8a9ad97f8a5 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:51.930958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1004/2030] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:51 2022] PUT /identity/v3/projects/227fd9f301d54b7196d00e9577632b52/users/b7698cbabf9e4721bdd9c16d1c0b5d58/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:51.942401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d0c39-42f3-4cde-b564-7abdd031b37e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:51.942701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d0c39-42f3-4cde-b564-7abdd031b37e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:51.942987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d0c39-42f3-4cde-b564-7abdd031b37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:51.945548 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1027/2031] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:51 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:51.949578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d221a-ce30-47e9-b4b7-7742f3553f1b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:51.980305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-d64d0c39-42f3-4cde-b564-7abdd031b37e None None] MFA Rules not processed for user `b7698cbabf9e4721bdd9c16d1c0b5d58`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:51.982505 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d64d0c39-42f3-4cde-b564-7abdd031b37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:52.019224 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:52.020481 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d221a-ce30-47e9-b4b7-7742f3553f1b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:52.025649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb7d23-8172-4f27-9f66-1ab7c492ab10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:52.026358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb7d23-8172-4f27-9f66-1ab7c492ab10 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:52.026640 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb7d23-8172-4f27-9f66-1ab7c492ab10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:52.026928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb7d23-8172-4f27-9f66-1ab7c492ab10 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:52.027606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb7d23-8172-4f27-9f66-1ab7c492ab10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:52.029581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb7d23-8172-4f27-9f66-1ab7c492ab10 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:52.051812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1028/2032] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:51 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:52.055688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb7d23-8172-4f27-9f66-1ab7c492ab10 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:52.058769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1005/2033] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:52.065905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb7d23-8172-4f27-9f66-1ab7c492ab10 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:52.069890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d0c39-42f3-4cde-b564-7abdd031b37e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:52.071042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69b15b-4b03-42fb-9f60-9b7cef72d7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:52.071260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d64d0c39-42f3-4cde-b564-7abdd031b37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:52.071698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69b15b-4b03-42fb-9f60-9b7cef72d7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:52.071978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69b15b-4b03-42fb-9f60-9b7cef72d7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:52.072257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69b15b-4b03-42fb-9f60-9b7cef72d7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:52.072989 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69b15b-4b03-42fb-9f60-9b7cef72d7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:52.074632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69b15b-4b03-42fb-9f60-9b7cef72d7e9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:52.084681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1029/2034] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:52.088785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69b15b-4b03-42fb-9f60-9b7cef72d7e9 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:52.098098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69b15b-4b03-42fb-9f60-9b7cef72d7e9 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:52.102724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991404b-120b-40d8-b2af-5897bb016a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:52.103566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991404b-120b-40d8-b2af-5897bb016a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:52.105410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991404b-120b-40d8-b2af-5897bb016a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:52.105720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991404b-120b-40d8-b2af-5897bb016a68 admin admin] PATH_INFO: `/v3/projects/a62f33fa1b654c66857e299406d30395/users/a1abed6791094849b017d5574ce2bed5/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:52.123588 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991404b-120b-40d8-b2af-5897bb016a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62f33fa1b654c66857e299406d30395, user_id=a1abed6791094849b017d5574ce2bed5, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:52.126317 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991404b-120b-40d8-b2af-5897bb016a68 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:52.129759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d0c39-42f3-4cde-b564-7abdd031b37e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:52.134615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9242573-285a-4f87-829c-fd45863ce1ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:52.135271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9242573-285a-4f87-829c-fd45863ce1ef service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:52.135884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9242573-285a-4f87-829c-fd45863ce1ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:52.135884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9242573-285a-4f87-829c-fd45863ce1ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:52.142371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1030/2035] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:52 2022] PUT /identity/v3/projects/a62f33fa1b654c66857e299406d30395/users/a1abed6791094849b017d5574ce2bed5/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:52.146656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991404b-120b-40d8-b2af-5897bb016a68 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:52.149325 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9242573-285a-4f87-829c-fd45863ce1ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:52.150794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9242573-285a-4f87-829c-fd45863ce1ef service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:52.165061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1006/2036] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:40:52.209072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:52.227725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991404b-120b-40d8-b2af-5897bb016a68 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:52.232924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d2b2a-2844-430e-8b12-52bc3c7621ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:52.233599 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d2b2a-2844-430e-8b12-52bc3c7621ca admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:52.233883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d2b2a-2844-430e-8b12-52bc3c7621ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:52.234188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d2b2a-2844-430e-8b12-52bc3c7621ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:52.235173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d2b2a-2844-430e-8b12-52bc3c7621ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:52.236752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d2b2a-2844-430e-8b12-52bc3c7621ca admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:52.246602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1031/2037] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:52.251306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9242573-285a-4f87-829c-fd45863ce1ef service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:52.261793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9242573-285a-4f87-829c-fd45863ce1ef service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:52.266923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad36111-28c4-456a-b4d2-8be193ad3a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:52.267819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad36111-28c4-456a-b4d2-8be193ad3a07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:52.268098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad36111-28c4-456a-b4d2-8be193ad3a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:52.268370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad36111-28c4-456a-b4d2-8be193ad3a07 admin admin] PATH_INFO: `/v3/projects/a62f33fa1b654c66857e299406d30395/users/a1abed6791094849b017d5574ce2bed5/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:52.270761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad36111-28c4-456a-b4d2-8be193ad3a07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62f33fa1b654c66857e299406d30395, user_id=a1abed6791094849b017d5574ce2bed5, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:52.273379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad36111-28c4-456a-b4d2-8be193ad3a07 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:52.284717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1007/2038] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:52 2022] PUT /identity/v3/projects/a62f33fa1b654c66857e299406d30395/users/a1abed6791094849b017d5574ce2bed5/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:52.298635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075aacd-21ab-46cc-a99b-d1aa1318ee07 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:52.299011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075aacd-21ab-46cc-a99b-d1aa1318ee07 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:52.299203 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075aacd-21ab-46cc-a99b-d1aa1318ee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:52.327201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1075aacd-21ab-46cc-a99b-d1aa1318ee07 None None] MFA Rules not processed for user `a1abed6791094849b017d5574ce2bed5`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:52.328501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1075aacd-21ab-46cc-a99b-d1aa1318ee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:52.414186 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1032/2039] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:52.424413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad36111-28c4-456a-b4d2-8be193ad3a07 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:52.502519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad36111-28c4-456a-b4d2-8be193ad3a07 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:52.506262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c006e-f933-4785-a1e4-0443dbbedbad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:52.506911 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c006e-f933-4785-a1e4-0443dbbedbad service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:52.507194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c006e-f933-4785-a1e4-0443dbbedbad service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:52.507591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c006e-f933-4785-a1e4-0443dbbedbad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:52.517198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c006e-f933-4785-a1e4-0443dbbedbad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:52.518655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c006e-f933-4785-a1e4-0443dbbedbad service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:52.529500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1008/2040] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:52.831560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075aacd-21ab-46cc-a99b-d1aa1318ee07 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:52.927573 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075aacd-21ab-46cc-a99b-d1aa1318ee07 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:52.936065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:52.936065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:52.936065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:52.936065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] PATH_INFO: `/v3/users/25750a71e10a4fb9ba9c80bb87179e26` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:52.937193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25750a71e10a4fb9ba9c80bb87179e26)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:52.939247 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:53.032504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25750a71e10a4fb9ba9c80bb87179e26'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.045396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25750a71e10a4fb9ba9c80bb87179e26'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.045973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25750a71e10a4fb9ba9c80bb87179e26'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.060589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25750a71e10a4fb9ba9c80bb87179e26'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.061547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25750a71e10a4fb9ba9c80bb87179e26'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.066967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:53.101270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1033/2041] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:52 2022] DELETE /identity/v3/users/25750a71e10a4fb9ba9c80bb87179e26 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:53.109118 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cc28a-2dcb-4e81-b85a-728d5153ae28 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:53.110249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cc28a-2dcb-4e81-b85a-728d5153ae28 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:53.110818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cc28a-2dcb-4e81-b85a-728d5153ae28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:53.157711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-609cc28a-2dcb-4e81-b85a-728d5153ae28 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:53.159762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-609cc28a-2dcb-4e81-b85a-728d5153ae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:53.255573 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:53.257181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:53.271508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1009/2042] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:53.287416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cc28a-2dcb-4e81-b85a-728d5153ae28 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:53.288895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-609cc28a-2dcb-4e81-b85a-728d5153ae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:53.295725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:53.347949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7d643-b80c-4b4e-97f9-1b06dcecc853 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:53.354215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:53.356289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:53.356656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:53.357707 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] PATH_INFO: `/v3/users/f7fefc97e5b441059c7916a9e7f838d7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:53.358666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7fefc97e5b441059c7916a9e7f838d7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:53.361031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:53.367835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cc28a-2dcb-4e81-b85a-728d5153ae28 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:53.373316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efca3f-6a20-4e97-aa38-a765200e927e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:53.374392 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efca3f-6a20-4e97-aa38-a765200e927e service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:53.374677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efca3f-6a20-4e97-aa38-a765200e927e service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:53.374956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efca3f-6a20-4e97-aa38-a765200e927e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:53.386529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efca3f-6a20-4e97-aa38-a765200e927e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:53.388602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efca3f-6a20-4e97-aa38-a765200e927e service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:53.403987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1010/2043] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:40:53.417449 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7fefc97e5b441059c7916a9e7f838d7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.425814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7fefc97e5b441059c7916a9e7f838d7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.426740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7fefc97e5b441059c7916a9e7f838d7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.436387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7fefc97e5b441059c7916a9e7f838d7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.437417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7fefc97e5b441059c7916a9e7f838d7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.463553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1034/2044] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:53 2022] DELETE /identity/v3/users/f7fefc97e5b441059c7916a9e7f838d7 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:53.467953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33794d-e6b6-444a-8c2b-b0e6a5d0eaca None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:53.468302 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33794d-e6b6-444a-8c2b-b0e6a5d0eaca None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:53.468535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33794d-e6b6-444a-8c2b-b0e6a5d0eaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:53.490719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-6a33794d-e6b6-444a-8c2b-b0e6a5d0eaca None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:53.492215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6a33794d-e6b6-444a-8c2b-b0e6a5d0eaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:53.625357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1011/2045] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:53.635397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:53.637045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:53.712534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095462f-8a1f-47a1-a891-537ec805d983 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:53.717848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd8b9e-4285-4f5f-985b-ce84f34f0bec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:53.718552 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd8b9e-4285-4f5f-985b-ce84f34f0bec service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:53.719069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd8b9e-4285-4f5f-985b-ce84f34f0bec service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:53.719176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd8b9e-4285-4f5f-985b-ce84f34f0bec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:53.730393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd8b9e-4285-4f5f-985b-ce84f34f0bec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:53.732429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd8b9e-4285-4f5f-985b-ce84f34f0bec service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:53.745675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1035/2046] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:53.837594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33794d-e6b6-444a-8c2b-b0e6a5d0eaca None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:53.839213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6a33794d-e6b6-444a-8c2b-b0e6a5d0eaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:53.857873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33794d-e6b6-444a-8c2b-b0e6a5d0eaca None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:53.863198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:53.864238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:53.864544 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:53.864906 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] PATH_INFO: `/v3/projects/7b23aca8b9e546e7af0ef5c2e150b8fe` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:53.866319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b23aca8b9e546e7af0ef5c2e150b8fe)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:53.868338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:53.924538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b23aca8b9e546e7af0ef5c2e150b8fe', 'request_id': 'req-4df60a8b-93ca-412c-a473-db17485b739a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.931333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b23aca8b9e546e7af0ef5c2e150b8fe', 'request_id': 'req-4df60a8b-93ca-412c-a473-db17485b739a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.937752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b23aca8b9e546e7af0ef5c2e150b8fe', 'request_id': 'req-4df60a8b-93ca-412c-a473-db17485b739a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.953064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] The token cache is being invalidate because project 7b23aca8b9e546e7af0ef5c2e150b8fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:53.953353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.954091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:53.955681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1012/2047] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:53 2022] DELETE /identity/v3/projects/7b23aca8b9e546e7af0ef5c2e150b8fe => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:53.959901 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd8b9e-4285-4f5f-985b-ce84f34f0bec service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:53.970147 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd8b9e-4285-4f5f-985b-ce84f34f0bec service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:53.973977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:54.037536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:54.038666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:54.039007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:54.039361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] PATH_INFO: `/v3/users/b7698cbabf9e4721bdd9c16d1c0b5d58` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:54.040652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7698cbabf9e4721bdd9c16d1c0b5d58)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:54.043041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:54.094214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7698cbabf9e4721bdd9c16d1c0b5d58'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.101708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7698cbabf9e4721bdd9c16d1c0b5d58'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.102518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7698cbabf9e4721bdd9c16d1c0b5d58'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.110799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7698cbabf9e4721bdd9c16d1c0b5d58'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.111936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7698cbabf9e4721bdd9c16d1c0b5d58'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.134694 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1036/2048] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:53 2022] DELETE /identity/v3/users/b7698cbabf9e4721bdd9c16d1c0b5d58 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:54.574727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:54.576242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:54.589010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df60a8b-93ca-412c-a473-db17485b739a admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:54.614677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:54.616447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:54.618763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:54.673238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ac770-9bca-46dc-bff9-5ba8d0efca31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:54.674264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ac770-9bca-46dc-bff9-5ba8d0efca31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:54.674553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ac770-9bca-46dc-bff9-5ba8d0efca31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:54.674936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ac770-9bca-46dc-bff9-5ba8d0efca31 admin admin] PATH_INFO: `/v3/projects/227fd9f301d54b7196d00e9577632b52` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:54.676721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ac770-9bca-46dc-bff9-5ba8d0efca31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=227fd9f301d54b7196d00e9577632b52)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:54.678589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ac770-9bca-46dc-bff9-5ba8d0efca31 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:54.703862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27287bfc-360a-4c6f-8313-12c718b20146 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:54.709556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:54.710457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:54.710740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:54.711012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] PATH_INFO: `/v3/projects/c5a44685abb34cf7833a66ae52330ae9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:54.712235 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5a44685abb34cf7833a66ae52330ae9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:54.714336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:54.743211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-239ac770-9bca-46dc-bff9-5ba8d0efca31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '227fd9f301d54b7196d00e9577632b52', 'request_id': 'req-239ac770-9bca-46dc-bff9-5ba8d0efca31'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.749490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:54.750547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-239ac770-9bca-46dc-bff9-5ba8d0efca31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '227fd9f301d54b7196d00e9577632b52', 'request_id': 'req-239ac770-9bca-46dc-bff9-5ba8d0efca31'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.757076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-239ac770-9bca-46dc-bff9-5ba8d0efca31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '227fd9f301d54b7196d00e9577632b52', 'request_id': 'req-239ac770-9bca-46dc-bff9-5ba8d0efca31'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.768632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5a44685abb34cf7833a66ae52330ae9', 'request_id': 'req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.776491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5a44685abb34cf7833a66ae52330ae9', 'request_id': 'req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.777693 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-239ac770-9bca-46dc-bff9-5ba8d0efca31 admin admin] The token cache is being invalidate because project 227fd9f301d54b7196d00e9577632b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:54.778729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-239ac770-9bca-46dc-bff9-5ba8d0efca31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.780037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-239ac770-9bca-46dc-bff9-5ba8d0efca31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.781940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1013/2049] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:54 2022] DELETE /identity/v3/projects/227fd9f301d54b7196d00e9577632b52 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:54.784418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5a44685abb34cf7833a66ae52330ae9', 'request_id': 'req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.786569 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347fe16-93c1-4897-bfef-8f746755a000 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:54.786990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347fe16-93c1-4897-bfef-8f746755a000 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:54.787338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347fe16-93c1-4897-bfef-8f746755a000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:54.796412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] The token cache is being invalidate because project c5a44685abb34cf7833a66ae52330ae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:54.796878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.797607 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.802953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1037/2050] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:54 2022] DELETE /identity/v3/projects/c5a44685abb34cf7833a66ae52330ae9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:54.804418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:54.806491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:54.808463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-5347fe16-93c1-4897-bfef-8f746755a000 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:54.809662 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5347fe16-93c1-4897-bfef-8f746755a000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:54.875682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1014/2051] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:54.883720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a5f93-9dc8-4eac-8868-6f852b75b489 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:54.884006 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a5f93-9dc8-4eac-8868-6f852b75b489 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:54.884282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a5f93-9dc8-4eac-8868-6f852b75b489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:54.889150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c7bbd-ae39-4f70-84ec-258b5cc9667c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:54.894106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:54.894984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:54.895264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:54.895536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] PATH_INFO: `/v3/users/a1abed6791094849b017d5574ce2bed5` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:54.896836 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1abed6791094849b017d5574ce2bed5)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:54.898807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:54.907550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-478a5f93-9dc8-4eac-8868-6f852b75b489 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:54.908778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-478a5f93-9dc8-4eac-8868-6f852b75b489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:54.917418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1015/2052] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:54.921276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a5f93-9dc8-4eac-8868-6f852b75b489 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:54.933772 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a5f93-9dc8-4eac-8868-6f852b75b489 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:54.939339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32be01-cb5a-4b13-b33b-189837d94201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:54.939952 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32be01-cb5a-4b13-b33b-189837d94201 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:54.940235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32be01-cb5a-4b13-b33b-189837d94201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:54.940510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32be01-cb5a-4b13-b33b-189837d94201 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:54.941248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32be01-cb5a-4b13-b33b-189837d94201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:54.941557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32be01-cb5a-4b13-b33b-189837d94201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:54.943238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32be01-cb5a-4b13-b33b-189837d94201 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:54.956877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1016/2053] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:40:54.959486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1abed6791094849b017d5574ce2bed5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.968622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1abed6791094849b017d5574ce2bed5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.969451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1abed6791094849b017d5574ce2bed5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.974548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32be01-cb5a-4b13-b33b-189837d94201 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:54.976263 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-3f32be01-cb5a-4b13-b33b-189837d94201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:54.977061 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1abed6791094849b017d5574ce2bed5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.977789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1abed6791094849b017d5574ce2bed5'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:54.989245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32be01-cb5a-4b13-b33b-189837d94201 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:54.994045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1518df2-3609-45be-a19f-e0cfd7b22952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:54.994701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1518df2-3609-45be-a19f-e0cfd7b22952 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:54.994989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1518df2-3609-45be-a19f-e0cfd7b22952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:54.995268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1518df2-3609-45be-a19f-e0cfd7b22952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:54.995787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1038/2054] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:40:54 2022] DELETE /identity/v3/users/a1abed6791094849b017d5574ce2bed5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:54.995966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1518df2-3609-45be-a19f-e0cfd7b22952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:54.998267 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1518df2-3609-45be-a19f-e0cfd7b22952 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.029239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1017/2055] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:54 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:55.033613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.074325 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:55.100913 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6f266-2ba6-4053-b033-4805c722a92f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.171758 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3445491-bb77-4bf2-9b02-12638a2f5607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.172530 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3445491-bb77-4bf2-9b02-12638a2f5607 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.172895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3445491-bb77-4bf2-9b02-12638a2f5607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.173805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3445491-bb77-4bf2-9b02-12638a2f5607 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.174710 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3445491-bb77-4bf2-9b02-12638a2f5607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.176716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3445491-bb77-4bf2-9b02-12638a2f5607 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.202312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1039/2056] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:55 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:55.206535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1518df2-3609-45be-a19f-e0cfd7b22952 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.217883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1518df2-3609-45be-a19f-e0cfd7b22952 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.223286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e8628-acd8-423d-a376-afc5bdd99892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.224042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e8628-acd8-423d-a376-afc5bdd99892 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.224331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e8628-acd8-423d-a376-afc5bdd99892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.224612 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e8628-acd8-423d-a376-afc5bdd99892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.225340 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e8628-acd8-423d-a376-afc5bdd99892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.226982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e8628-acd8-423d-a376-afc5bdd99892 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.237781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1018/2057] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:55.241697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3445491-bb77-4bf2-9b02-12638a2f5607 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.252729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3445491-bb77-4bf2-9b02-12638a2f5607 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.258045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ad7fd-c63f-4622-8bdd-792021a28406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.258666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ad7fd-c63f-4622-8bdd-792021a28406 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.258975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ad7fd-c63f-4622-8bdd-792021a28406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.259263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ad7fd-c63f-4622-8bdd-792021a28406 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.259984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ad7fd-c63f-4622-8bdd-792021a28406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.261595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ad7fd-c63f-4622-8bdd-792021a28406 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.272807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1040/2058] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:55.275032 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e8628-acd8-423d-a376-afc5bdd99892 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.276784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ad7fd-c63f-4622-8bdd-792021a28406 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.277011 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ab5e8628-acd8-423d-a376-afc5bdd99892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:55.287060 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ad7fd-c63f-4622-8bdd-792021a28406 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.291772 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb14337-7f1c-4762-b6b1-4c1ac68720f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.292698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb14337-7f1c-4762-b6b1-4c1ac68720f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.292986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb14337-7f1c-4762-b6b1-4c1ac68720f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.293257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb14337-7f1c-4762-b6b1-4c1ac68720f0 admin admin] PATH_INFO: `/v3/projects/0c11ee26e9b54109a3521928dc6c072e/users/734170cc2fc74aecbe7fd4ad9e76c97f/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.293957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e8628-acd8-423d-a376-afc5bdd99892 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.300011 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ada1d-dfc0-4349-9da2-63a448243b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.302406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ada1d-dfc0-4349-9da2-63a448243b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.302406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ada1d-dfc0-4349-9da2-63a448243b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.302406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ada1d-dfc0-4349-9da2-63a448243b93 admin admin] PATH_INFO: `/v3/projects/a62f33fa1b654c66857e299406d30395` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.303420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ada1d-dfc0-4349-9da2-63a448243b93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a62f33fa1b654c66857e299406d30395)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.305512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ada1d-dfc0-4349-9da2-63a448243b93 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.307244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:55.313260 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb14337-7f1c-4762-b6b1-4c1ac68720f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c11ee26e9b54109a3521928dc6c072e, user_id=734170cc2fc74aecbe7fd4ad9e76c97f, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.318180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb14337-7f1c-4762-b6b1-4c1ac68720f0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.337102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1041/2059] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:55 2022] PUT /identity/v3/projects/0c11ee26e9b54109a3521928dc6c072e/users/734170cc2fc74aecbe7fd4ad9e76c97f/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:55.350748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fa249-3ae1-4111-8a0c-e8ac839e0f89 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.351146 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fa249-3ae1-4111-8a0c-e8ac839e0f89 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.351713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fa249-3ae1-4111-8a0c-e8ac839e0f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.378194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-505ada1d-dfc0-4349-9da2-63a448243b93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a62f33fa1b654c66857e299406d30395', 'request_id': 'req-505ada1d-dfc0-4349-9da2-63a448243b93'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:55.385599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-505ada1d-dfc0-4349-9da2-63a448243b93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a62f33fa1b654c66857e299406d30395', 'request_id': 'req-505ada1d-dfc0-4349-9da2-63a448243b93'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:55.392681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-505ada1d-dfc0-4349-9da2-63a448243b93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a62f33fa1b654c66857e299406d30395', 'request_id': 'req-505ada1d-dfc0-4349-9da2-63a448243b93'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:55.412998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-505ada1d-dfc0-4349-9da2-63a448243b93 admin admin] The token cache is being invalidate because project a62f33fa1b654c66857e299406d30395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:40:55.413257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-505ada1d-dfc0-4349-9da2-63a448243b93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:55.414037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-505ada1d-dfc0-4349-9da2-63a448243b93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:40:55.416666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1019/2060] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:40:55 2022] DELETE /identity/v3/projects/a62f33fa1b654c66857e299406d30395 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:55.421693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b02fa249-3ae1-4111-8a0c-e8ac839e0f89 None None] MFA Rules not processed for user `734170cc2fc74aecbe7fd4ad9e76c97f`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:55.422997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b02fa249-3ae1-4111-8a0c-e8ac839e0f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:55.423809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d076f-4f37-4ab9-82f7-6557b1524a67 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.425296 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d076f-4f37-4ab9-82f7-6557b1524a67 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.425296 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d076f-4f37-4ab9-82f7-6557b1524a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.456136 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-ba5d076f-4f37-4ab9-82f7-6557b1524a67 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:55.457378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ba5d076f-4f37-4ab9-82f7-6557b1524a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:55.534838 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1042/2061] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:55.539394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fa249-3ae1-4111-8a0c-e8ac839e0f89 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.540784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b02fa249-3ae1-4111-8a0c-e8ac839e0f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:55.553426 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1020/2062] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:55.560694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eef2c-96e6-4ac8-8c6b-0abc3947e766 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.561453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eef2c-96e6-4ac8-8c6b-0abc3947e766 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.561783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eef2c-96e6-4ac8-8c6b-0abc3947e766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.575035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:55.583317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-000eef2c-96e6-4ac8-8c6b-0abc3947e766 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:55.584539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-000eef2c-96e6-4ac8-8c6b-0abc3947e766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:55.593212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1021/2063] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:55.598007 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000eef2c-96e6-4ac8-8c6b-0abc3947e766 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.607828 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000eef2c-96e6-4ac8-8c6b-0abc3947e766 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.613495 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18c6bb-12c8-4184-8234-de0d385a8f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.614178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18c6bb-12c8-4184-8234-de0d385a8f78 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.614458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18c6bb-12c8-4184-8234-de0d385a8f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.614735 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18c6bb-12c8-4184-8234-de0d385a8f78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.615482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18c6bb-12c8-4184-8234-de0d385a8f78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:40:55.615785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18c6bb-12c8-4184-8234-de0d385a8f78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.617360 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18c6bb-12c8-4184-8234-de0d385a8f78 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.620080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fa249-3ae1-4111-8a0c-e8ac839e0f89 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.624896 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4ab30-f28b-4f78-8211-8ee277f1a2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.625634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4ab30-f28b-4f78-8211-8ee277f1a2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.626127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4ab30-f28b-4f78-8211-8ee277f1a2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.626475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4ab30-f28b-4f78-8211-8ee277f1a2a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.627243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4ab30-f28b-4f78-8211-8ee277f1a2a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.629320 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1022/2064] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:40:55.630214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4ab30-f28b-4f78-8211-8ee277f1a2a8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.633543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18c6bb-12c8-4184-8234-de0d385a8f78 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.644368 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18c6bb-12c8-4184-8234-de0d385a8f78 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.649447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848ab41-bf7e-4b03-a965-e49be2c7f52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.650209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848ab41-bf7e-4b03-a965-e49be2c7f52d admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.650565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848ab41-bf7e-4b03-a965-e49be2c7f52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.650858 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848ab41-bf7e-4b03-a965-e49be2c7f52d admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.651573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848ab41-bf7e-4b03-a965-e49be2c7f52d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.654981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848ab41-bf7e-4b03-a965-e49be2c7f52d admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.663000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1043/2065] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:55 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:55.666724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4ab30-f28b-4f78-8211-8ee277f1a2a8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.688254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1023/2066] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:55 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:55.692190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848ab41-bf7e-4b03-a965-e49be2c7f52d admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.750184 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4ab30-f28b-4f78-8211-8ee277f1a2a8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.772684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:55.793149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848ab41-bf7e-4b03-a965-e49be2c7f52d admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.798861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e925e-4152-4b09-9fe8-1f28f0c7defa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.799660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e925e-4152-4b09-9fe8-1f28f0c7defa admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.800020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e925e-4152-4b09-9fe8-1f28f0c7defa admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.800367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e925e-4152-4b09-9fe8-1f28f0c7defa admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.801139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e925e-4152-4b09-9fe8-1f28f0c7defa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.804449 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e925e-4152-4b09-9fe8-1f28f0c7defa admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.833878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1024/2067] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:55 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:55.841317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e925e-4152-4b09-9fe8-1f28f0c7defa admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.853227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e925e-4152-4b09-9fe8-1f28f0c7defa admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.858646 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bd909-d844-4a75-af7b-bd86526ade65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.859124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bd909-d844-4a75-af7b-bd86526ade65 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.859817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bd909-d844-4a75-af7b-bd86526ade65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.860175 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bd909-d844-4a75-af7b-bd86526ade65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.860872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bd909-d844-4a75-af7b-bd86526ade65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.862514 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bd909-d844-4a75-af7b-bd86526ade65 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.863180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568fdb7-c311-4568-8171-114795f07b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.864115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568fdb7-c311-4568-8171-114795f07b6b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.864505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568fdb7-c311-4568-8171-114795f07b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.865200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568fdb7-c311-4568-8171-114795f07b6b admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.866010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568fdb7-c311-4568-8171-114795f07b6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.868679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568fdb7-c311-4568-8171-114795f07b6b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.874660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1025/2068] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:55.878424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bd909-d844-4a75-af7b-bd86526ade65 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.888826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bd909-d844-4a75-af7b-bd86526ade65 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.894670 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8daaef-e194-41f4-a6d6-e3f6b38d166e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.895264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8daaef-e194-41f4-a6d6-e3f6b38d166e admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.895543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8daaef-e194-41f4-a6d6-e3f6b38d166e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.895812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8daaef-e194-41f4-a6d6-e3f6b38d166e admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.896548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8daaef-e194-41f4-a6d6-e3f6b38d166e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.898141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8daaef-e194-41f4-a6d6-e3f6b38d166e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.900066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1044/2069] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:55 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:55.904292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568fdb7-c311-4568-8171-114795f07b6b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.909741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1026/2070] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:55.913781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8daaef-e194-41f4-a6d6-e3f6b38d166e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.914222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568fdb7-c311-4568-8171-114795f07b6b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.918914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d706ef-532c-4d30-9dee-4b88419555f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.919559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d706ef-532c-4d30-9dee-4b88419555f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.919843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d706ef-532c-4d30-9dee-4b88419555f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.920116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d706ef-532c-4d30-9dee-4b88419555f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.920843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d706ef-532c-4d30-9dee-4b88419555f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.922457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d706ef-532c-4d30-9dee-4b88419555f4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.923573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8daaef-e194-41f4-a6d6-e3f6b38d166e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.928210 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaaf285-0e7f-47a6-94b6-efb038ee089b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.929072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaaf285-0e7f-47a6-94b6-efb038ee089b admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.929353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaaf285-0e7f-47a6-94b6-efb038ee089b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.929633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaaf285-0e7f-47a6-94b6-efb038ee089b admin admin] PATH_INFO: `/v3/projects/3fe0a286aca64c789b36e330bd231a99/users/0acf5c291d714d69b88f6a0def9aeeaa/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.933136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1045/2071] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:55.936948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d706ef-532c-4d30-9dee-4b88419555f4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.944323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaaf285-0e7f-47a6-94b6-efb038ee089b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe0a286aca64c789b36e330bd231a99, user_id=0acf5c291d714d69b88f6a0def9aeeaa, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.946524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d706ef-532c-4d30-9dee-4b88419555f4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.947022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaaf285-0e7f-47a6-94b6-efb038ee089b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.952259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34fa01-0e92-4c60-9ed4-eb588ba9aaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:55.953151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34fa01-0e92-4c60-9ed4-eb588ba9aaff admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.953595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34fa01-0e92-4c60-9ed4-eb588ba9aaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.954145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34fa01-0e92-4c60-9ed4-eb588ba9aaff admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.954913 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34fa01-0e92-4c60-9ed4-eb588ba9aaff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:55.956858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34fa01-0e92-4c60-9ed4-eb588ba9aaff admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:55.961153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1027/2072] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:55 2022] PUT /identity/v3/projects/3fe0a286aca64c789b36e330bd231a99/users/0acf5c291d714d69b88f6a0def9aeeaa/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:55.966777 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622cdeb-c7b9-4129-ae00-b8cc3ab1ffcb None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:55.967072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622cdeb-c7b9-4129-ae00-b8cc3ab1ffcb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:55.967351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622cdeb-c7b9-4129-ae00-b8cc3ab1ffcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:55.968757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1046/2073] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:40:55.972490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34fa01-0e92-4c60-9ed4-eb588ba9aaff admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:55.983597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34fa01-0e92-4c60-9ed4-eb588ba9aaff admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:55.987578 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b622cdeb-c7b9-4129-ae00-b8cc3ab1ffcb None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:55.988799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b622cdeb-c7b9-4129-ae00-b8cc3ab1ffcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:56.011332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:56.051001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df265ec2-657c-442b-b063-11e13b45046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:56.052293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df265ec2-657c-442b-b063-11e13b45046b admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:56.052680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df265ec2-657c-442b-b063-11e13b45046b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:56.053063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df265ec2-657c-442b-b063-11e13b45046b admin admin] PATH_INFO: `/v3/projects/a25b87b29e5444bbb1d7d28e7b242042/users/419117b2311a4792a3d3c2f43c013bb9/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:56.064613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1028/2074] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:40:56.069003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df265ec2-657c-442b-b063-11e13b45046b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25b87b29e5444bbb1d7d28e7b242042, user_id=419117b2311a4792a3d3c2f43c013bb9, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:56.071772 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df265ec2-657c-442b-b063-11e13b45046b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:56.073951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622cdeb-c7b9-4129-ae00-b8cc3ab1ffcb None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:56.076070 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b622cdeb-c7b9-4129-ae00-b8cc3ab1ffcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:56.088126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1047/2075] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:55 2022] PUT /identity/v3/projects/a25b87b29e5444bbb1d7d28e7b242042/users/419117b2311a4792a3d3c2f43c013bb9/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:40:56.098482 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f72c70-9976-4254-a3e5-74304a4b71f0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:56.098840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f72c70-9976-4254-a3e5-74304a4b71f0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:56.099190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f72c70-9976-4254-a3e5-74304a4b71f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:56.128664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-34f72c70-9976-4254-a3e5-74304a4b71f0 None None] MFA Rules not processed for user `419117b2311a4792a3d3c2f43c013bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:56.130068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-34f72c70-9976-4254-a3e5-74304a4b71f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:56.141610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622cdeb-c7b9-4129-ae00-b8cc3ab1ffcb None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:56.193454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:56.199005 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893085c-327f-4c53-8771-f3efe69dbea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:56.200414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893085c-327f-4c53-8771-f3efe69dbea6 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:56.200882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893085c-327f-4c53-8771-f3efe69dbea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:56.201156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893085c-327f-4c53-8771-f3efe69dbea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:56.207293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1048/2076] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:56.211844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f72c70-9976-4254-a3e5-74304a4b71f0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:56.262069 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893085c-327f-4c53-8771-f3efe69dbea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:56.264158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893085c-327f-4c53-8771-f3efe69dbea6 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:56.273656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1029/2077] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:40:56.274946 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f72c70-9976-4254-a3e5-74304a4b71f0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:56.279682 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589a544-49df-4f04-883b-29a175ae1fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:56.280412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589a544-49df-4f04-883b-29a175ae1fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:56.280771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589a544-49df-4f04-883b-29a175ae1fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:56.281121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589a544-49df-4f04-883b-29a175ae1fb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:56.281892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589a544-49df-4f04-883b-29a175ae1fb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:56.283964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589a544-49df-4f04-883b-29a175ae1fb8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:56.313683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1049/2078] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:56 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:56.317947 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893085c-327f-4c53-8771-f3efe69dbea6 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:56.328775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893085c-327f-4c53-8771-f3efe69dbea6 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:56.392749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96dfec-bdc1-4ef5-8205-debad8fead40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:56.393687 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96dfec-bdc1-4ef5-8205-debad8fead40 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:56.394131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96dfec-bdc1-4ef5-8205-debad8fead40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:56.394672 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96dfec-bdc1-4ef5-8205-debad8fead40 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:56.395606 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96dfec-bdc1-4ef5-8205-debad8fead40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:56.397929 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96dfec-bdc1-4ef5-8205-debad8fead40 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:56.424009 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1030/2079] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:56 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:40:56.427987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589a544-49df-4f04-883b-29a175ae1fb8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:56.449565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589a544-49df-4f04-883b-29a175ae1fb8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:56.455818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60458c-03a0-4bf0-886d-375d379980c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:56.456500 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60458c-03a0-4bf0-886d-375d379980c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:56.456783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60458c-03a0-4bf0-886d-375d379980c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:56.457190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60458c-03a0-4bf0-886d-375d379980c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:56.458129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f60458c-03a0-4bf0-886d-375d379980c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:56.459793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f60458c-03a0-4bf0-886d-375d379980c9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:56.473958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1050/2080] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:56.482097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96dfec-bdc1-4ef5-8205-debad8fead40 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:56.498388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96dfec-bdc1-4ef5-8205-debad8fead40 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:56.504018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d38fb-2331-4c5e-9754-b2324b3b15e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:56.505035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d38fb-2331-4c5e-9754-b2324b3b15e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:56.505413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d38fb-2331-4c5e-9754-b2324b3b15e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:56.505785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d38fb-2331-4c5e-9754-b2324b3b15e0 admin admin] PATH_INFO: `/v3/projects/761687ece8004846b240d08eaf0c00a3/users/7ce95fe8fc6c4d5faaae9a567936f1e9/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:56.526236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d38fb-2331-4c5e-9754-b2324b3b15e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=761687ece8004846b240d08eaf0c00a3, user_id=7ce95fe8fc6c4d5faaae9a567936f1e9, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:56.529235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d38fb-2331-4c5e-9754-b2324b3b15e0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:56.546943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1031/2081] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:56 2022] PUT /identity/v3/projects/761687ece8004846b240d08eaf0c00a3/users/7ce95fe8fc6c4d5faaae9a567936f1e9/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:56.551645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60458c-03a0-4bf0-886d-375d379980c9 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:56.565523 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60458c-03a0-4bf0-886d-375d379980c9 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:56.576453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:56.633465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f165ec6-b289-483b-928a-162f53247369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:56.634411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f165ec6-b289-483b-928a-162f53247369 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:56.634794 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f165ec6-b289-483b-928a-162f53247369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:56.635196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f165ec6-b289-483b-928a-162f53247369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:56.636022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f165ec6-b289-483b-928a-162f53247369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:56.637904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f165ec6-b289-483b-928a-162f53247369 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:56.653515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1051/2082] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:56.654948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d38fb-2331-4c5e-9754-b2324b3b15e0 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:56.665591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d38fb-2331-4c5e-9754-b2324b3b15e0 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:56.671124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0965b59-ae33-431a-a844-a8cc5215e1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:56.672125 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0965b59-ae33-431a-a844-a8cc5215e1bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:56.672507 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0965b59-ae33-431a-a844-a8cc5215e1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:56.672886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0965b59-ae33-431a-a844-a8cc5215e1bd admin admin] PATH_INFO: `/v3/projects/761687ece8004846b240d08eaf0c00a3/users/7ce95fe8fc6c4d5faaae9a567936f1e9/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:56.674996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0965b59-ae33-431a-a844-a8cc5215e1bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=761687ece8004846b240d08eaf0c00a3, user_id=7ce95fe8fc6c4d5faaae9a567936f1e9, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:56.677751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0965b59-ae33-431a-a844-a8cc5215e1bd admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:56.689743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1032/2083] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:56 2022] PUT /identity/v3/projects/761687ece8004846b240d08eaf0c00a3/users/7ce95fe8fc6c4d5faaae9a567936f1e9/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:56.701162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64af281-1cd0-4841-9de1-76d6a89e4fbc None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:56.701444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64af281-1cd0-4841-9de1-76d6a89e4fbc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:56.701724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64af281-1cd0-4841-9de1-76d6a89e4fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:56.755861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c64af281-1cd0-4841-9de1-76d6a89e4fbc None None] MFA Rules not processed for user `7ce95fe8fc6c4d5faaae9a567936f1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:56.757284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c64af281-1cd0-4841-9de1-76d6a89e4fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:56.839740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1052/2084] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:56.852372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0965b59-ae33-431a-a844-a8cc5215e1bd admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:56.854520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e0965b59-ae33-431a-a844-a8cc5215e1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:56.924369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0965b59-ae33-431a-a844-a8cc5215e1bd admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:56.929725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe96991-3918-426b-a363-0ecc0d53373e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:56.930523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe96991-3918-426b-a363-0ecc0d53373e service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:56.931400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe96991-3918-426b-a363-0ecc0d53373e service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:56.931882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe96991-3918-426b-a363-0ecc0d53373e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:56.937741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:56.997112 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe96991-3918-426b-a363-0ecc0d53373e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:56.998668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe96991-3918-426b-a363-0ecc0d53373e service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:57.012089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1033/2085] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:57.055915 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5835b2-3f4e-4a6d-9043-1ed9bbfebf76 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:57.056376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5835b2-3f4e-4a6d-9043-1ed9bbfebf76 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:57.056547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5835b2-3f4e-4a6d-9043-1ed9bbfebf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:57.093495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2a5835b2-3f4e-4a6d-9043-1ed9bbfebf76 None None] MFA Rules not processed for user `0acf5c291d714d69b88f6a0def9aeeaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:57.094770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2a5835b2-3f4e-4a6d-9043-1ed9bbfebf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:57.187838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe96991-3918-426b-a363-0ecc0d53373e service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:57.188273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1053/2086] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:57.288893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe96991-3918-426b-a363-0ecc0d53373e service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:57.294700 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2ae01-d27b-404f-8669-48435c0d17ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:57.295443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2ae01-d27b-404f-8669-48435c0d17ef admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:57.295691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2ae01-d27b-404f-8669-48435c0d17ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:57.296079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2ae01-d27b-404f-8669-48435c0d17ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:57.296955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2ae01-d27b-404f-8669-48435c0d17ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:57.298912 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2ae01-d27b-404f-8669-48435c0d17ef admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:57.332202 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1034/2087] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:40:57 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:57.338166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5835b2-3f4e-4a6d-9043-1ed9bbfebf76 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:57.349044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5835b2-3f4e-4a6d-9043-1ed9bbfebf76 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:57.372629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:57.414976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb763233-56f4-4c5f-92cc-f7728cd6521b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:57.416081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb763233-56f4-4c5f-92cc-f7728cd6521b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:57.416081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb763233-56f4-4c5f-92cc-f7728cd6521b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:57.416276 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb763233-56f4-4c5f-92cc-f7728cd6521b admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:57.416883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb763233-56f4-4c5f-92cc-f7728cd6521b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:57.418977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb763233-56f4-4c5f-92cc-f7728cd6521b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:57.439607 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1054/2088] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:40:57 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:40:57.444085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2ae01-d27b-404f-8669-48435c0d17ef admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:57.459505 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2ae01-d27b-404f-8669-48435c0d17ef admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:57.464230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ffe4b-6306-4524-a60f-3b8d93fa7fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:57.464949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ffe4b-6306-4524-a60f-3b8d93fa7fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:57.465252 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ffe4b-6306-4524-a60f-3b8d93fa7fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:57.466190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ffe4b-6306-4524-a60f-3b8d93fa7fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:57.466914 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ffe4b-6306-4524-a60f-3b8d93fa7fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:57.468627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ffe4b-6306-4524-a60f-3b8d93fa7fb2 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:57.479222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1035/2089] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:57.483998 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb763233-56f4-4c5f-92cc-f7728cd6521b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:57.494853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb763233-56f4-4c5f-92cc-f7728cd6521b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:57.504146 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28bd87-bf0f-4dc2-b68c-7c235e178639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:57.505041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28bd87-bf0f-4dc2-b68c-7c235e178639 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:57.505351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28bd87-bf0f-4dc2-b68c-7c235e178639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:57.505631 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28bd87-bf0f-4dc2-b68c-7c235e178639 admin admin] PATH_INFO: `/v3/projects/5edca2f0196a4c67a57aaab43a1a7825/users/65aa58a021c04157a51406337000f475/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:57.522610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28bd87-bf0f-4dc2-b68c-7c235e178639 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5edca2f0196a4c67a57aaab43a1a7825, user_id=65aa58a021c04157a51406337000f475, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:57.525461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28bd87-bf0f-4dc2-b68c-7c235e178639 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:57.539818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1055/2090] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:57 2022] PUT /identity/v3/projects/5edca2f0196a4c67a57aaab43a1a7825/users/65aa58a021c04157a51406337000f475/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:57.543818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ffe4b-6306-4524-a60f-3b8d93fa7fb2 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:57.553589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ffe4b-6306-4524-a60f-3b8d93fa7fb2 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:57.608376 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8acdc3-6af4-4660-b586-d4637610f613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:57.609035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8acdc3-6af4-4660-b586-d4637610f613 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:57.609320 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8acdc3-6af4-4660-b586-d4637610f613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:57.609856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8acdc3-6af4-4660-b586-d4637610f613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:57.610610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8acdc3-6af4-4660-b586-d4637610f613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:57.612209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8acdc3-6af4-4660-b586-d4637610f613 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:57.622443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1036/2091] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:40:57.626611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28bd87-bf0f-4dc2-b68c-7c235e178639 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:57.636871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28bd87-bf0f-4dc2-b68c-7c235e178639 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:57.642255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464cbe4-9485-48ca-be45-4db4f06dd863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:57.643132 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464cbe4-9485-48ca-be45-4db4f06dd863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:57.643415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464cbe4-9485-48ca-be45-4db4f06dd863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:57.643690 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464cbe4-9485-48ca-be45-4db4f06dd863 admin admin] PATH_INFO: `/v3/projects/5edca2f0196a4c67a57aaab43a1a7825/users/65aa58a021c04157a51406337000f475/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:57.645694 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464cbe4-9485-48ca-be45-4db4f06dd863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5edca2f0196a4c67a57aaab43a1a7825, user_id=65aa58a021c04157a51406337000f475, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:57.648395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464cbe4-9485-48ca-be45-4db4f06dd863 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:57.658569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1056/2092] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:40:57 2022] PUT /identity/v3/projects/5edca2f0196a4c67a57aaab43a1a7825/users/65aa58a021c04157a51406337000f475/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:40:58.291664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371b2c9-bdea-4e8c-ab95-231b5a402679 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:58.292093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371b2c9-bdea-4e8c-ab95-231b5a402679 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:58.292214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371b2c9-bdea-4e8c-ab95-231b5a402679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:58.321879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-5371b2c9-bdea-4e8c-ab95-231b5a402679 None None] MFA Rules not processed for user `65aa58a021c04157a51406337000f475`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:40:58.323117 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5371b2c9-bdea-4e8c-ab95-231b5a402679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:40:58.327221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:40:58.400215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1037/2093] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:40:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:40:58.412138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464cbe4-9485-48ca-be45-4db4f06dd863 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:58.493288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464cbe4-9485-48ca-be45-4db4f06dd863 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:58.498731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b02fb2-5c93-4d94-9671-c00e729d4eba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:58.499388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b02fb2-5c93-4d94-9671-c00e729d4eba service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:58.500262 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b02fb2-5c93-4d94-9671-c00e729d4eba service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:58.500544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b02fb2-5c93-4d94-9671-c00e729d4eba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:58.589943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b02fb2-5c93-4d94-9671-c00e729d4eba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:58.592108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b02fb2-5c93-4d94-9671-c00e729d4eba service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:58.601905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1057/2094] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:40:59.698349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371b2c9-bdea-4e8c-ab95-231b5a402679 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:40:59.708542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371b2c9-bdea-4e8c-ab95-231b5a402679 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:40:59.712215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900c933-d2e9-4b98-9e3e-64417eba10f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:40:59.712865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900c933-d2e9-4b98-9e3e-64417eba10f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:40:59.713148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900c933-d2e9-4b98-9e3e-64417eba10f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:40:59.713454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900c933-d2e9-4b98-9e3e-64417eba10f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:40:59.771469 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900c933-d2e9-4b98-9e3e-64417eba10f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:40:59.772906 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900c933-d2e9-4b98-9e3e-64417eba10f8 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:40:59.782352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1038/2095] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:40:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:00.327693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b02fb2-5c93-4d94-9671-c00e729d4eba service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:00.336762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b02fb2-5c93-4d94-9671-c00e729d4eba service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:00.341201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fce81-260c-4cea-b8df-b7fd07083c7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:00.341839 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fce81-260c-4cea-b8df-b7fd07083c7f service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:00.342157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fce81-260c-4cea-b8df-b7fd07083c7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:00.342439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fce81-260c-4cea-b8df-b7fd07083c7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:00.371650 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:00.404568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fce81-260c-4cea-b8df-b7fd07083c7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:00.406038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fce81-260c-4cea-b8df-b7fd07083c7f service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:00.415736 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1058/2096] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:01.029309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900c933-d2e9-4b98-9e3e-64417eba10f8 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:01.030772 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0900c933-d2e9-4b98-9e3e-64417eba10f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:01.041607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900c933-d2e9-4b98-9e3e-64417eba10f8 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:01.045741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c6696-cba6-4e7c-ad3f-d08a7bd15407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:01.046403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c6696-cba6-4e7c-ad3f-d08a7bd15407 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:01.046687 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c6696-cba6-4e7c-ad3f-d08a7bd15407 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:01.046964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c6696-cba6-4e7c-ad3f-d08a7bd15407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:01.055819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c6696-cba6-4e7c-ad3f-d08a7bd15407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:01.057223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c6696-cba6-4e7c-ad3f-d08a7bd15407 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:01.067463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1039/2097] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:01 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:05.414867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fce81-260c-4cea-b8df-b7fd07083c7f service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:05.416423 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-678fce81-260c-4cea-b8df-b7fd07083c7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:05.486243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fce81-260c-4cea-b8df-b7fd07083c7f service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:05.491006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba23220-bf81-46fe-af48-06bc99bd6c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:05.491882 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba23220-bf81-46fe-af48-06bc99bd6c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:05.492164 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba23220-bf81-46fe-af48-06bc99bd6c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:05.492870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba23220-bf81-46fe-af48-06bc99bd6c32 admin admin] PATH_INFO: `/v3/users/37d3ed0e5f0f43ee8a1404b7ab856cd9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:05.493736 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba23220-bf81-46fe-af48-06bc99bd6c32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37d3ed0e5f0f43ee8a1404b7ab856cd9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:05.495873 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba23220-bf81-46fe-af48-06bc99bd6c32 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:05.558040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bba23220-bf81-46fe-af48-06bc99bd6c32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37d3ed0e5f0f43ee8a1404b7ab856cd9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:05.570308 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bba23220-bf81-46fe-af48-06bc99bd6c32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37d3ed0e5f0f43ee8a1404b7ab856cd9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:05.570943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bba23220-bf81-46fe-af48-06bc99bd6c32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37d3ed0e5f0f43ee8a1404b7ab856cd9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:05.580775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c6696-cba6-4e7c-ad3f-d08a7bd15407 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:05.582211 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bba23220-bf81-46fe-af48-06bc99bd6c32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37d3ed0e5f0f43ee8a1404b7ab856cd9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:05.583224 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bba23220-bf81-46fe-af48-06bc99bd6c32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37d3ed0e5f0f43ee8a1404b7ab856cd9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:05.593942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c6696-cba6-4e7c-ad3f-d08a7bd15407 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:05.600266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b50ec2-a19d-47d1-a093-945a88da1e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:05.601311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b50ec2-a19d-47d1-a093-945a88da1e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:05.601867 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b50ec2-a19d-47d1-a093-945a88da1e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:05.602239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b50ec2-a19d-47d1-a093-945a88da1e17 admin admin] PATH_INFO: `/v3/users/734170cc2fc74aecbe7fd4ad9e76c97f` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:05.603980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b50ec2-a19d-47d1-a093-945a88da1e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=734170cc2fc74aecbe7fd4ad9e76c97f)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:05.606666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b50ec2-a19d-47d1-a093-945a88da1e17 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:05.620273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1059/2098] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:05 2022] DELETE /identity/v3/users/37d3ed0e5f0f43ee8a1404b7ab856cd9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:05.622072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c6dd0-c357-4a70-8cb2-4f50c1e1a8c2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:05.622366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c6dd0-c357-4a70-8cb2-4f50c1e1a8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:05.622646 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c6dd0-c357-4a70-8cb2-4f50c1e1a8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:05.649347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-064c6dd0-c357-4a70-8cb2-4f50c1e1a8c2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:05.650714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-064c6dd0-c357-4a70-8cb2-4f50c1e1a8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:05.669457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-10b50ec2-a19d-47d1-a093-945a88da1e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '734170cc2fc74aecbe7fd4ad9e76c97f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:05.670214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-10b50ec2-a19d-47d1-a093-945a88da1e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '734170cc2fc74aecbe7fd4ad9e76c97f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:05.670921 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-10b50ec2-a19d-47d1-a093-945a88da1e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '734170cc2fc74aecbe7fd4ad9e76c97f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:05.678777 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-10b50ec2-a19d-47d1-a093-945a88da1e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '734170cc2fc74aecbe7fd4ad9e76c97f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:05.687659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-10b50ec2-a19d-47d1-a093-945a88da1e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '734170cc2fc74aecbe7fd4ad9e76c97f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:05.698780 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:05.704263 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:05.716219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1040/2099] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:05 2022] DELETE /identity/v3/users/734170cc2fc74aecbe7fd4ad9e76c97f => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:41:05.720465 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0a1a6-0bd8-4f2b-9a02-cbebfbdbb59b None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:05.720761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0a1a6-0bd8-4f2b-9a02-cbebfbdbb59b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:05.721038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0a1a6-0bd8-4f2b-9a02-cbebfbdbb59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:05.729213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1060/2100] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:05.739126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c6dd0-c357-4a70-8cb2-4f50c1e1a8c2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:05.740561 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-064c6dd0-c357-4a70-8cb2-4f50c1e1a8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:05.741392 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-86d0a1a6-0bd8-4f2b-9a02-cbebfbdbb59b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:05.742685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-86d0a1a6-0bd8-4f2b-9a02-cbebfbdbb59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:05.802429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c6dd0-c357-4a70-8cb2-4f50c1e1a8c2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:05.811670 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1041/2101] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:05.827165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0a1a6-0bd8-4f2b-9a02-cbebfbdbb59b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:05.860378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c8912-df1a-48e1-85eb-2cf44d458609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:05.861144 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c8912-df1a-48e1-85eb-2cf44d458609 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:05.861669 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c8912-df1a-48e1-85eb-2cf44d458609 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:05.862279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c8912-df1a-48e1-85eb-2cf44d458609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:05.879679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c8912-df1a-48e1-85eb-2cf44d458609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:05.881139 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c8912-df1a-48e1-85eb-2cf44d458609 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:05.893808 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1061/2102] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:41:05.925540 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0a1a6-0bd8-4f2b-9a02-cbebfbdbb59b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:05.929629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb28e12-0edc-4ef2-9826-737df9835f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:05.930314 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb28e12-0edc-4ef2-9826-737df9835f61 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:05.930594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb28e12-0edc-4ef2-9826-737df9835f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:05.930866 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb28e12-0edc-4ef2-9826-737df9835f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:05.946359 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb28e12-0edc-4ef2-9826-737df9835f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:05.947796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb28e12-0edc-4ef2-9826-737df9835f61 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:05.959796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1042/2103] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:06.489713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c8912-df1a-48e1-85eb-2cf44d458609 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:06.492112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ce5c8912-df1a-48e1-85eb-2cf44d458609 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:06.517892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c8912-df1a-48e1-85eb-2cf44d458609 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:06.528296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea62d4a-3d15-4602-8b34-16615201472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:06.532868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb28e12-0edc-4ef2-9826-737df9835f61 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:06.533019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea62d4a-3d15-4602-8b34-16615201472b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:06.533349 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea62d4a-3d15-4602-8b34-16615201472b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:06.533875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea62d4a-3d15-4602-8b34-16615201472b admin admin] PATH_INFO: `/v3/projects/1d46d4698d3d43c2a15a6b40f08090af` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:06.535489 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-bcb28e12-0edc-4ef2-9826-737df9835f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:06.536069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea62d4a-3d15-4602-8b34-16615201472b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d46d4698d3d43c2a15a6b40f08090af)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:06.538326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea62d4a-3d15-4602-8b34-16615201472b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:06.556443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb28e12-0edc-4ef2-9826-737df9835f61 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:06.562449 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:06.564157 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:06.564817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:06.565272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] PATH_INFO: `/v3/projects/0c11ee26e9b54109a3521928dc6c072e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:06.567280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c11ee26e9b54109a3521928dc6c072e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:06.570616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:06.614154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-eea62d4a-3d15-4602-8b34-16615201472b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d46d4698d3d43c2a15a6b40f08090af', 'request_id': 'req-eea62d4a-3d15-4602-8b34-16615201472b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.623511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-eea62d4a-3d15-4602-8b34-16615201472b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d46d4698d3d43c2a15a6b40f08090af', 'request_id': 'req-eea62d4a-3d15-4602-8b34-16615201472b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.632841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-eea62d4a-3d15-4602-8b34-16615201472b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d46d4698d3d43c2a15a6b40f08090af', 'request_id': 'req-eea62d4a-3d15-4602-8b34-16615201472b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.640953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c11ee26e9b54109a3521928dc6c072e', 'request_id': 'req-ed7abb53-9ddf-4757-b41f-637d718165a0'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.645504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-eea62d4a-3d15-4602-8b34-16615201472b admin admin] The token cache is being invalidate because project 1d46d4698d3d43c2a15a6b40f08090af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:06.645928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-eea62d4a-3d15-4602-8b34-16615201472b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.646703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-eea62d4a-3d15-4602-8b34-16615201472b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.647493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c11ee26e9b54109a3521928dc6c072e', 'request_id': 'req-ed7abb53-9ddf-4757-b41f-637d718165a0'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.649091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1062/2104] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:06 2022] DELETE /identity/v3/projects/1d46d4698d3d43c2a15a6b40f08090af => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:06.653906 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c11ee26e9b54109a3521928dc6c072e', 'request_id': 'req-ed7abb53-9ddf-4757-b41f-637d718165a0'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.654634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2ebb8-6f50-4b01-94e7-2e0a1ba9c062 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:06.655033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2ebb8-6f50-4b01-94e7-2e0a1ba9c062 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:06.655355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2ebb8-6f50-4b01-94e7-2e0a1ba9c062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:06.670420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] The token cache is being invalidate because project 0c11ee26e9b54109a3521928dc6c072e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:06.670997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.671819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.677050 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1043/2105] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:06 2022] DELETE /identity/v3/projects/0c11ee26e9b54109a3521928dc6c072e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:06.678373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7bf2ebb8-6f50-4b01-94e7-2e0a1ba9c062 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:06.678490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:06.679732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7bf2ebb8-6f50-4b01-94e7-2e0a1ba9c062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:06.680298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:06.707516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:06.709776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:06.757466 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1063/2106] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:06.758653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7abb53-9ddf-4757-b41f-637d718165a0 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:06.763332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:06.764238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:06.764497 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:06.764769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] PATH_INFO: `/v3/users/419117b2311a4792a3d3c2f43c013bb9` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:06.765161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab0552-af19-468f-97c5-eb200657b434 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:06.765453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab0552-af19-468f-97c5-eb200657b434 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:06.765733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab0552-af19-468f-97c5-eb200657b434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:06.766161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] RBAC: Authorizing `identity:delete_user(user_id=419117b2311a4792a3d3c2f43c013bb9)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:06.768072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:06.788856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-cbab0552-af19-468f-97c5-eb200657b434 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:06.790129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cbab0552-af19-468f-97c5-eb200657b434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:06.802119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1064/2107] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:41:06.805662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab0552-af19-468f-97c5-eb200657b434 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:06.816831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab0552-af19-468f-97c5-eb200657b434 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:06.822275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05184e-921c-4994-a228-0fdc8d8875b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:06.823035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05184e-921c-4994-a228-0fdc8d8875b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:06.823386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05184e-921c-4994-a228-0fdc8d8875b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:06.823740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05184e-921c-4994-a228-0fdc8d8875b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:06.824584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05184e-921c-4994-a228-0fdc8d8875b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:06.824965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05184e-921c-4994-a228-0fdc8d8875b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:06.826681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05184e-921c-4994-a228-0fdc8d8875b5 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:06.828980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '419117b2311a4792a3d3c2f43c013bb9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.829903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '419117b2311a4792a3d3c2f43c013bb9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.830790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '419117b2311a4792a3d3c2f43c013bb9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.839807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '419117b2311a4792a3d3c2f43c013bb9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.844703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1065/2108] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:06.845593 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05184e-921c-4994-a228-0fdc8d8875b5 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:06.847830 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ae05184e-921c-4994-a228-0fdc8d8875b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:06.850408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '419117b2311a4792a3d3c2f43c013bb9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:06.862660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05184e-921c-4994-a228-0fdc8d8875b5 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:06.867862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705d32c-3792-4b7a-8faf-33d73b0946bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:06.868680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705d32c-3792-4b7a-8faf-33d73b0946bc admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:06.869082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705d32c-3792-4b7a-8faf-33d73b0946bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:06.869461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705d32c-3792-4b7a-8faf-33d73b0946bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:06.870293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705d32c-3792-4b7a-8faf-33d73b0946bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:06.872819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1044/2109] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:06 2022] DELETE /identity/v3/users/419117b2311a4792a3d3c2f43c013bb9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:06.872968 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705d32c-3792-4b7a-8faf-33d73b0946bc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:06.903199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1066/2110] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:06 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:06.907421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:06.997797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1119571-8ca7-43ed-b3e2-c162ef6e881e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:07.074027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:07.089712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbada579-dcd4-4ef6-a091-81b2803e1448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:07.091238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbada579-dcd4-4ef6-a091-81b2803e1448 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:07.094040 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbada579-dcd4-4ef6-a091-81b2803e1448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:07.094444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbada579-dcd4-4ef6-a091-81b2803e1448 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:07.095341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbada579-dcd4-4ef6-a091-81b2803e1448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:07.097467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbada579-dcd4-4ef6-a091-81b2803e1448 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:07.127826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1045/2111] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:06 2022] POST /identity/v3/users => generated 292 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:07.132999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705d32c-3792-4b7a-8faf-33d73b0946bc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:07.147106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705d32c-3792-4b7a-8faf-33d73b0946bc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:07.152447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae69445-65fa-411e-99da-a4e53e8fe623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:07.153267 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae69445-65fa-411e-99da-a4e53e8fe623 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:07.153634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae69445-65fa-411e-99da-a4e53e8fe623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:07.154055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae69445-65fa-411e-99da-a4e53e8fe623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:07.154875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae69445-65fa-411e-99da-a4e53e8fe623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:07.156625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae69445-65fa-411e-99da-a4e53e8fe623 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:07.167514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1067/2112] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:07.176488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbada579-dcd4-4ef6-a091-81b2803e1448 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:07.192622 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbada579-dcd4-4ef6-a091-81b2803e1448 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:07.197845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b683d83-9f10-4514-8a20-9eda903d28b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:07.198907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b683d83-9f10-4514-8a20-9eda903d28b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:07.198907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b683d83-9f10-4514-8a20-9eda903d28b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:07.199097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b683d83-9f10-4514-8a20-9eda903d28b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:07.199855 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b683d83-9f10-4514-8a20-9eda903d28b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:07.201666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b683d83-9f10-4514-8a20-9eda903d28b6 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:07.217958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1046/2113] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:07.219844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae69445-65fa-411e-99da-a4e53e8fe623 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:07.233730 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae69445-65fa-411e-99da-a4e53e8fe623 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:07.236405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b683d83-9f10-4514-8a20-9eda903d28b6 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:07.238328 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7b683d83-9f10-4514-8a20-9eda903d28b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:07.240934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11e8d7-f878-46ae-8559-aa6ee7425b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:07.240934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11e8d7-f878-46ae-8559-aa6ee7425b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:07.240934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11e8d7-f878-46ae-8559-aa6ee7425b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:07.241246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11e8d7-f878-46ae-8559-aa6ee7425b51 admin admin] PATH_INFO: `/v3/projects/83acd2cd92644556a4fc2d132be84ed1/users/dd718a8bf2cb4325ad7d29131ea47f48/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:07.256348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b683d83-9f10-4514-8a20-9eda903d28b6 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:07.263401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:07.264276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:07.264561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:07.264843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] PATH_INFO: `/v3/projects/a25b87b29e5444bbb1d7d28e7b242042` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:07.266420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25b87b29e5444bbb1d7d28e7b242042)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:07.268345 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:07.269158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11e8d7-f878-46ae-8559-aa6ee7425b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83acd2cd92644556a4fc2d132be84ed1, user_id=dd718a8bf2cb4325ad7d29131ea47f48, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:07.272558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11e8d7-f878-46ae-8559-aa6ee7425b51 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:07.291592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1068/2114] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:07 2022] PUT /identity/v3/projects/83acd2cd92644556a4fc2d132be84ed1/users/dd718a8bf2cb4325ad7d29131ea47f48/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:07.302849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b12c83-06f5-4fcf-9d43-228fa0b71e5a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:07.303396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b12c83-06f5-4fcf-9d43-228fa0b71e5a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:07.303858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b12c83-06f5-4fcf-9d43-228fa0b71e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:07.338370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25b87b29e5444bbb1d7d28e7b242042', 'request_id': 'req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:07.342884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-86b12c83-06f5-4fcf-9d43-228fa0b71e5a None None] MFA Rules not processed for user `dd718a8bf2cb4325ad7d29131ea47f48`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:07.344155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-86b12c83-06f5-4fcf-9d43-228fa0b71e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:07.346221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25b87b29e5444bbb1d7d28e7b242042', 'request_id': 'req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:07.352775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25b87b29e5444bbb1d7d28e7b242042', 'request_id': 'req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:07.372121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] The token cache is being invalidate because project a25b87b29e5444bbb1d7d28e7b242042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:07.372412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:07.373519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:07.375987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1047/2115] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:07 2022] DELETE /identity/v3/projects/a25b87b29e5444bbb1d7d28e7b242042 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:41:07.379876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:07.381819 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:07.450754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1069/2116] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:41:07.451163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b12c83-06f5-4fcf-9d43-228fa0b71e5a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:07.452751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-86b12c83-06f5-4fcf-9d43-228fa0b71e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:07.460868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332ddd5-a5cc-4fb8-8c2c-774ab3cef163 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:07.472143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3295ab-07ab-4e87-924e-df3a05244f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:07.473268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3295ab-07ab-4e87-924e-df3a05244f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:07.474228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3295ab-07ab-4e87-924e-df3a05244f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:07.474524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3295ab-07ab-4e87-924e-df3a05244f27 admin admin] PATH_INFO: `/v3/users/7ce95fe8fc6c4d5faaae9a567936f1e9` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:07.475793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3295ab-07ab-4e87-924e-df3a05244f27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce95fe8fc6c4d5faaae9a567936f1e9)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:07.477854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3295ab-07ab-4e87-924e-df3a05244f27 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:07.509439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:07.526271 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b12c83-06f5-4fcf-9d43-228fa0b71e5a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:07.531092 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5f9f8-16ea-4671-a3ee-a6445e30af14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:07.531520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-3e3295ab-07ab-4e87-924e-df3a05244f27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce95fe8fc6c4d5faaae9a567936f1e9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:07.532148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5f9f8-16ea-4671-a3ee-a6445e30af14 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:07.532268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-3e3295ab-07ab-4e87-924e-df3a05244f27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce95fe8fc6c4d5faaae9a567936f1e9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:07.532458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5f9f8-16ea-4671-a3ee-a6445e30af14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:07.532831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-3e3295ab-07ab-4e87-924e-df3a05244f27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce95fe8fc6c4d5faaae9a567936f1e9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:07.532930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5f9f8-16ea-4671-a3ee-a6445e30af14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:07.535204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-00c5f9f8-16ea-4671-a3ee-a6445e30af14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:07.538883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:07.541182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-3e3295ab-07ab-4e87-924e-df3a05244f27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce95fe8fc6c4d5faaae9a567936f1e9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:07.550477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5f9f8-16ea-4671-a3ee-a6445e30af14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:07.552132 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5f9f8-16ea-4671-a3ee-a6445e30af14 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:07.552812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-3e3295ab-07ab-4e87-924e-df3a05244f27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce95fe8fc6c4d5faaae9a567936f1e9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:07.562802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1070/2117] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:41:07.573326 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1048/2118] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:07 2022] DELETE /identity/v3/users/7ce95fe8fc6c4d5faaae9a567936f1e9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:07.907774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5f9f8-16ea-4671-a3ee-a6445e30af14 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:07.911229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-00c5f9f8-16ea-4671-a3ee-a6445e30af14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:07.931507 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5f9f8-16ea-4671-a3ee-a6445e30af14 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:08.000899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998f368-0135-4e1a-a334-d62ab476451c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:08.002023 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f368-0135-4e1a-a334-d62ab476451c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:08.002310 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f368-0135-4e1a-a334-d62ab476451c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:08.002590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f368-0135-4e1a-a334-d62ab476451c admin admin] PATH_INFO: `/v3/projects/761687ece8004846b240d08eaf0c00a3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:08.003804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f368-0135-4e1a-a334-d62ab476451c admin admin] RBAC: Authorizing `identity:delete_project(project_id=761687ece8004846b240d08eaf0c00a3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:08.005739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f368-0135-4e1a-a334-d62ab476451c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:08.054390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6998f368-0135-4e1a-a334-d62ab476451c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '761687ece8004846b240d08eaf0c00a3', 'request_id': 'req-6998f368-0135-4e1a-a334-d62ab476451c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:08.065309 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6998f368-0135-4e1a-a334-d62ab476451c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '761687ece8004846b240d08eaf0c00a3', 'request_id': 'req-6998f368-0135-4e1a-a334-d62ab476451c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:08.072424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6998f368-0135-4e1a-a334-d62ab476451c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '761687ece8004846b240d08eaf0c00a3', 'request_id': 'req-6998f368-0135-4e1a-a334-d62ab476451c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:08.083637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6998f368-0135-4e1a-a334-d62ab476451c admin admin] The token cache is being invalidate because project 761687ece8004846b240d08eaf0c00a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:08.083926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6998f368-0135-4e1a-a334-d62ab476451c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:08.084562 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6998f368-0135-4e1a-a334-d62ab476451c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:08.086155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1071/2119] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:07 2022] DELETE /identity/v3/projects/761687ece8004846b240d08eaf0c00a3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:08.091000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395ee14-3be9-483c-9bfa-156f2c8f53ef None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:08.091437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395ee14-3be9-483c-9bfa-156f2c8f53ef None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:08.091590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395ee14-3be9-483c-9bfa-156f2c8f53ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:08.111310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f395ee14-3be9-483c-9bfa-156f2c8f53ef None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:08.112602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f395ee14-3be9-483c-9bfa-156f2c8f53ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:08.174473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1049/2120] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:08.182530 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8ad73-2867-491c-b192-903a21bf0c24 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:08.182982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8ad73-2867-491c-b192-903a21bf0c24 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:08.183292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8ad73-2867-491c-b192-903a21bf0c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:08.204670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-03e8ad73-2867-491c-b192-903a21bf0c24 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:08.206061 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-03e8ad73-2867-491c-b192-903a21bf0c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:08.214289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1072/2121] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:08.218798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395ee14-3be9-483c-9bfa-156f2c8f53ef None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:08.228867 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395ee14-3be9-483c-9bfa-156f2c8f53ef None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:08.233674 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f9be7-2875-4264-8b04-958cf7fd7865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:08.234667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f9be7-2875-4264-8b04-958cf7fd7865 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:08.234954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f9be7-2875-4264-8b04-958cf7fd7865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:08.235235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f9be7-2875-4264-8b04-958cf7fd7865 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:08.235964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f9be7-2875-4264-8b04-958cf7fd7865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:08.236274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f9be7-2875-4264-8b04-958cf7fd7865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:08.238013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f9be7-2875-4264-8b04-958cf7fd7865 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:08.253743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1050/2122] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:08.254877 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8ad73-2867-491c-b192-903a21bf0c24 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:08.264754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8ad73-2867-491c-b192-903a21bf0c24 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:08.269521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d7ea0-27f3-45b6-97d7-80342f9a061f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:08.270211 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d7ea0-27f3-45b6-97d7-80342f9a061f admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:08.270498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d7ea0-27f3-45b6-97d7-80342f9a061f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:08.270776 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d7ea0-27f3-45b6-97d7-80342f9a061f admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:08.271685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d7ea0-27f3-45b6-97d7-80342f9a061f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:08.273643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d7ea0-27f3-45b6-97d7-80342f9a061f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:08.301953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1073/2123] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:08 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:08.306219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f9be7-2875-4264-8b04-958cf7fd7865 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:08.317437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f9be7-2875-4264-8b04-958cf7fd7865 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:08.375998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5823f9-2e10-4704-8907-db8bbacee5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:08.376685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5823f9-2e10-4704-8907-db8bbacee5cb admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:08.377049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5823f9-2e10-4704-8907-db8bbacee5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:08.377236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5823f9-2e10-4704-8907-db8bbacee5cb admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:08.378034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5823f9-2e10-4704-8907-db8bbacee5cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:08.379950 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5823f9-2e10-4704-8907-db8bbacee5cb admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:08.401262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1051/2124] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:08 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:08.405353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d7ea0-27f3-45b6-97d7-80342f9a061f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:08.416063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d7ea0-27f3-45b6-97d7-80342f9a061f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:08.421400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a945d1-9843-43dc-95e0-1d49cb5487a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:08.422099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a945d1-9843-43dc-95e0-1d49cb5487a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:08.422388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a945d1-9843-43dc-95e0-1d49cb5487a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:08.422668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a945d1-9843-43dc-95e0-1d49cb5487a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:08.423395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a945d1-9843-43dc-95e0-1d49cb5487a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:08.424999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a945d1-9843-43dc-95e0-1d49cb5487a4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:08.438250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1074/2125] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:08.442999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5823f9-2e10-4704-8907-db8bbacee5cb admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:08.453292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5823f9-2e10-4704-8907-db8bbacee5cb admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:08.458095 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4781d26-a560-44c5-b9e3-c87e6ebaaf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:08.458762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4781d26-a560-44c5-b9e3-c87e6ebaaf33 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:08.459075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4781d26-a560-44c5-b9e3-c87e6ebaaf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:08.459353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4781d26-a560-44c5-b9e3-c87e6ebaaf33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:08.460085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4781d26-a560-44c5-b9e3-c87e6ebaaf33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:08.461696 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4781d26-a560-44c5-b9e3-c87e6ebaaf33 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:08.472692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1052/2126] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:08.477001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a945d1-9843-43dc-95e0-1d49cb5487a4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:08.488081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a945d1-9843-43dc-95e0-1d49cb5487a4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:08.492732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c274c-16be-4eaa-9458-60cb80e28cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:08.493622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c274c-16be-4eaa-9458-60cb80e28cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:08.493906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c274c-16be-4eaa-9458-60cb80e28cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:08.494334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c274c-16be-4eaa-9458-60cb80e28cbe admin admin] PATH_INFO: `/v3/projects/147e44123a2b40caa56a9871ca92b3e9/users/820f31be79d240e5bac59fdd54748fbf/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:08.514613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c274c-16be-4eaa-9458-60cb80e28cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=147e44123a2b40caa56a9871ca92b3e9, user_id=820f31be79d240e5bac59fdd54748fbf, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:08.517307 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c274c-16be-4eaa-9458-60cb80e28cbe admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:08.532950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1075/2127] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:08 2022] PUT /identity/v3/projects/147e44123a2b40caa56a9871ca92b3e9/users/820f31be79d240e5bac59fdd54748fbf/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:41:08.543797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff45ea-af45-474b-98e7-38be6f70fc0c None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:08.544089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff45ea-af45-474b-98e7-38be6f70fc0c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:08.544368 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff45ea-af45-474b-98e7-38be6f70fc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:08.573159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-19ff45ea-af45-474b-98e7-38be6f70fc0c None None] MFA Rules not processed for user `820f31be79d240e5bac59fdd54748fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:08.574432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-19ff45ea-af45-474b-98e7-38be6f70fc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:08.602092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:08.642465 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1053/2128] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:08 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:08.654175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c274c-16be-4eaa-9458-60cb80e28cbe admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:08.656486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-997c274c-16be-4eaa-9458-60cb80e28cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:08.670491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:08.723298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c274c-16be-4eaa-9458-60cb80e28cbe admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:08.727123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e487b2-654b-4cea-93ae-ad2eef27e9fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b07dcd8c0de4d1694d4363520726df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:08.727773 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e487b2-654b-4cea-93ae-ad2eef27e9fc service glance] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:08.728060 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e487b2-654b-4cea-93ae-ad2eef27e9fc service glance] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:08.728339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e487b2-654b-4cea-93ae-ad2eef27e9fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:08.738024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e487b2-654b-4cea-93ae-ad2eef27e9fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:08.739576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e487b2-654b-4cea-93ae-ad2eef27e9fc service glance] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:08.749450 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1076/2129] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:18.639414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=15 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:18.744533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=22 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:20.609774 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff45ea-af45-474b-98e7-38be6f70fc0c None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:20.612104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-19ff45ea-af45-474b-98e7-38be6f70fc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:20.700364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff45ea-af45-474b-98e7-38be6f70fc0c None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:20.704961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:20.705842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:20.706156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:20.706430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] PATH_INFO: `/v3/users/dd718a8bf2cb4325ad7d29131ea47f48` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:20.707628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd718a8bf2cb4325ad7d29131ea47f48)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:20.709543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:20.764596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd718a8bf2cb4325ad7d29131ea47f48'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:20.765191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd718a8bf2cb4325ad7d29131ea47f48'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:20.765804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd718a8bf2cb4325ad7d29131ea47f48'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:20.773176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd718a8bf2cb4325ad7d29131ea47f48'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:20.782416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd718a8bf2cb4325ad7d29131ea47f48'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:20.801617 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1054/2130] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:20 2022] DELETE /identity/v3/users/dd718a8bf2cb4325ad7d29131ea47f48 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:20.806417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d94086-539a-4b69-89de-373fde9cff26 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:20.806962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d94086-539a-4b69-89de-373fde9cff26 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:20.806962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d94086-539a-4b69-89de-373fde9cff26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:20.829076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c5d94086-539a-4b69-89de-373fde9cff26 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:20.830356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c5d94086-539a-4b69-89de-373fde9cff26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:20.902964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1077/2131] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:20.914242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:20.915966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:20.998055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bc3fe-9799-4cce-8e0f-ea557f9e6ae5 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:21.015297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9136a-4cca-4e7c-91d5-cc37d83a6301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:21.015923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9136a-4cca-4e7c-91d5-cc37d83a6301 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:21.016227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9136a-4cca-4e7c-91d5-cc37d83a6301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:21.016511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9136a-4cca-4e7c-91d5-cc37d83a6301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:21.035489 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9136a-4cca-4e7c-91d5-cc37d83a6301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:21.037692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9136a-4cca-4e7c-91d5-cc37d83a6301 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:21.051051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1055/2132] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:21.930008 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d94086-539a-4b69-89de-373fde9cff26 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:21.934659 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c5d94086-539a-4b69-89de-373fde9cff26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:21.947287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d94086-539a-4b69-89de-373fde9cff26 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:21.953686 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:21.954604 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:21.954887 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:21.955163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d admin admin] PATH_INFO: `/v3/projects/83acd2cd92644556a4fc2d132be84ed1` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:21.956445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=83acd2cd92644556a4fc2d132be84ed1)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:21.958849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:22.047550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83acd2cd92644556a4fc2d132be84ed1', 'request_id': 'req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:22.060829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83acd2cd92644556a4fc2d132be84ed1', 'request_id': 'req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:22.075083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef43d-9926-4707-aede-b086c3f4ae4e None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.075981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef43d-9926-4707-aede-b086c3f4ae4e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.076435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef43d-9926-4707-aede-b086c3f4ae4e None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.078416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1056/2133] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:41:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:41:22.079100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83acd2cd92644556a4fc2d132be84ed1', 'request_id': 'req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:22.103880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70743353-689d-44d2-a218-99c13f9e2995 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.104341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70743353-689d-44d2-a218-99c13f9e2995 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.104766 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70743353-689d-44d2-a218-99c13f9e2995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.128996 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d admin admin] The token cache is being invalidate because project 83acd2cd92644556a4fc2d132be84ed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:22.129270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:22.129975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b0ed96b7-c6d4-4bd1-81fb-2eec8798da2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:22.131577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1078/2134] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:21 2022] DELETE /identity/v3/projects/83acd2cd92644556a4fc2d132be84ed1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:22.137402 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0067b-9c8d-4a5e-a7eb-82f5fe13e1bd None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.137758 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0067b-9c8d-4a5e-a7eb-82f5fe13e1bd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.138055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0067b-9c8d-4a5e-a7eb-82f5fe13e1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.148685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-70743353-689d-44d2-a218-99c13f9e2995 None None] MFA Rules not processed for user `81037f61e01e4a93982503720f7404e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:22.150351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-70743353-689d-44d2-a218-99c13f9e2995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:22.161012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-45b0067b-9c8d-4a5e-a7eb-82f5fe13e1bd None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:22.162330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-45b0067b-9c8d-4a5e-a7eb-82f5fe13e1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:22.175418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:22.277851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1057/2135] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:22.298860 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70743353-689d-44d2-a218-99c13f9e2995 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:22.299070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1079/2136] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:22.299167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-70743353-689d-44d2-a218-99c13f9e2995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:22.300630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a75db-e827-43a7-93a2-9a503c8f8b55 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.300924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a75db-e827-43a7-93a2-9a503c8f8b55 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.301206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a75db-e827-43a7-93a2-9a503c8f8b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.318923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70743353-689d-44d2-a218-99c13f9e2995 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:22.328426 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cafe6-6271-4f73-a2dc-713340914d71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:22.329441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cafe6-6271-4f73-a2dc-713340914d71 service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.329904 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cafe6-6271-4f73-a2dc-713340914d71 service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.330047 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3f7a75db-e827-43a7-93a2-9a503c8f8b55 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:22.330473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cafe6-6271-4f73-a2dc-713340914d71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.331243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3f7a75db-e827-43a7-93a2-9a503c8f8b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:22.342866 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cafe6-6271-4f73-a2dc-713340914d71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:22.345279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cafe6-6271-4f73-a2dc-713340914d71 service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:22.360164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1058/2137] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:22.420997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1080/2138] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:22.425498 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cafe6-6271-4f73-a2dc-713340914d71 service placement] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:22.446186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cafe6-6271-4f73-a2dc-713340914d71 service placement] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:22.455153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941abf42-9012-48cf-bda1-748d939afb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:22.456344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941abf42-9012-48cf-bda1-748d939afb4e admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.456851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941abf42-9012-48cf-bda1-748d939afb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.457361 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941abf42-9012-48cf-bda1-748d939afb4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.458607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941abf42-9012-48cf-bda1-748d939afb4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:22.459243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941abf42-9012-48cf-bda1-748d939afb4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:22.461743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941abf42-9012-48cf-bda1-748d939afb4e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:22.479846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1059/2139] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:22.485326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a75db-e827-43a7-93a2-9a503c8f8b55 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:22.498433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a75db-e827-43a7-93a2-9a503c8f8b55 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:22.504851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48353999-09af-48c6-b6b0-f82146bd6255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:22.505700 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48353999-09af-48c6-b6b0-f82146bd6255 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.506128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48353999-09af-48c6-b6b0-f82146bd6255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.506501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48353999-09af-48c6-b6b0-f82146bd6255 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.507299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48353999-09af-48c6-b6b0-f82146bd6255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:22.509538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48353999-09af-48c6-b6b0-f82146bd6255 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:22.559609 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:22.590393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1081/2140] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:22 2022] POST /identity/v3/projects => generated 359 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:22.595730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941abf42-9012-48cf-bda1-748d939afb4e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:22.607536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941abf42-9012-48cf-bda1-748d939afb4e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:22.703729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689d775-d5a7-4bfc-b56f-7bdc527f58a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:22.704785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689d775-d5a7-4bfc-b56f-7bdc527f58a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.705522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689d775-d5a7-4bfc-b56f-7bdc527f58a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.706331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689d775-d5a7-4bfc-b56f-7bdc527f58a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.709016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689d775-d5a7-4bfc-b56f-7bdc527f58a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:22.714852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689d775-d5a7-4bfc-b56f-7bdc527f58a4 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:22.760788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1060/2141] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:22 2022] POST /identity/v3/users => generated 280 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:22.763373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48353999-09af-48c6-b6b0-f82146bd6255 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:22.777465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48353999-09af-48c6-b6b0-f82146bd6255 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:22.784768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3e1f8-d4fa-4b36-ba8e-6db2732b5ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:22.785486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3e1f8-d4fa-4b36-ba8e-6db2732b5ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.785951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3e1f8-d4fa-4b36-ba8e-6db2732b5ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.786228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3e1f8-d4fa-4b36-ba8e-6db2732b5ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.788171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3e1f8-d4fa-4b36-ba8e-6db2732b5ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:22.788610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3e1f8-d4fa-4b36-ba8e-6db2732b5ec5 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:22.802890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1082/2142] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:22.806543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689d775-d5a7-4bfc-b56f-7bdc527f58a4 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:22.817543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689d775-d5a7-4bfc-b56f-7bdc527f58a4 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:22.822836 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964a8a1-157c-40ee-aeda-1823695b9cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:22.823523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964a8a1-157c-40ee-aeda-1823695b9cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.824123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964a8a1-157c-40ee-aeda-1823695b9cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.824123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964a8a1-157c-40ee-aeda-1823695b9cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.824790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964a8a1-157c-40ee-aeda-1823695b9cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:22.827668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964a8a1-157c-40ee-aeda-1823695b9cc9 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:22.846290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1061/2143] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:22.848048 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3e1f8-d4fa-4b36-ba8e-6db2732b5ec5 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:22.860295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3e1f8-d4fa-4b36-ba8e-6db2732b5ec5 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:22.866403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95eb53-36e1-4c9b-b08a-458ac3a52ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:22.867328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95eb53-36e1-4c9b-b08a-458ac3a52ba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.867610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95eb53-36e1-4c9b-b08a-458ac3a52ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.867890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95eb53-36e1-4c9b-b08a-458ac3a52ba7 admin admin] PATH_INFO: `/v3/projects/6276528cc9f2420d9f400a986160ae37/users/4705861fb1b4457cbc6f6f5d603bf939/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.893940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95eb53-36e1-4c9b-b08a-458ac3a52ba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6276528cc9f2420d9f400a986160ae37, user_id=4705861fb1b4457cbc6f6f5d603bf939, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:22.896870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95eb53-36e1-4c9b-b08a-458ac3a52ba7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:22.925846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1083/2144] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:22 2022] PUT /identity/v3/projects/6276528cc9f2420d9f400a986160ae37/users/4705861fb1b4457cbc6f6f5d603bf939/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:41:22.931332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f4e20-b5e9-4583-bd76-deef6b768d87 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.932054 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f4e20-b5e9-4583-bd76-deef6b768d87 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.932054 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f4e20-b5e9-4583-bd76-deef6b768d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.964634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-414f4e20-b5e9-4583-bd76-deef6b768d87 None None] MFA Rules not processed for user `4705861fb1b4457cbc6f6f5d603bf939`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:22.967047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-414f4e20-b5e9-4583-bd76-deef6b768d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:22.971923 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ee044-be7d-4034-b073-8247b30f3ab0 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:22.972823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ee044-be7d-4034-b073-8247b30f3ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:22.973321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ee044-be7d-4034-b073-8247b30f3ab0 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:22.976385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1084/2145] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:41:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:41:23.026082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06db7a-216c-436d-bf68-29a1a0054fd2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:23.027390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06db7a-216c-436d-bf68-29a1a0054fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:23.027979 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06db7a-216c-436d-bf68-29a1a0054fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:23.060789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9f06db7a-216c-436d-bf68-29a1a0054fd2 None None] MFA Rules not processed for user `81037f61e01e4a93982503720f7404e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:23.062415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9f06db7a-216c-436d-bf68-29a1a0054fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:23.086114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:23.096803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1062/2146] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:23.113270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f4e20-b5e9-4583-bd76-deef6b768d87 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:23.125126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-414f4e20-b5e9-4583-bd76-deef6b768d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:23.236005 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f4e20-b5e9-4583-bd76-deef6b768d87 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:23.237322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:23.238574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:23.239432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:23.240609 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] PATH_INFO: `/v3/users/0acf5c291d714d69b88f6a0def9aeeaa` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:23.242413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0acf5c291d714d69b88f6a0def9aeeaa)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:23.244644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:23.264677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1085/2147] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:41:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:23.266053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06db7a-216c-436d-bf68-29a1a0054fd2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:23.267688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9f06db7a-216c-436d-bf68-29a1a0054fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:23.340820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0acf5c291d714d69b88f6a0def9aeeaa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:23.341526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0acf5c291d714d69b88f6a0def9aeeaa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:23.342184 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0acf5c291d714d69b88f6a0def9aeeaa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:23.350857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0acf5c291d714d69b88f6a0def9aeeaa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:23.364909 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0acf5c291d714d69b88f6a0def9aeeaa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:23.392453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06db7a-216c-436d-bf68-29a1a0054fd2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:23.397695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a96a5-e4d0-409a-b8ed-da0fde1d8e27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:23.398817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a96a5-e4d0-409a-b8ed-da0fde1d8e27 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:23.399107 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a96a5-e4d0-409a-b8ed-da0fde1d8e27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:23.399390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a96a5-e4d0-409a-b8ed-da0fde1d8e27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:23.400886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1063/2148] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:23 2022] DELETE /identity/v3/users/0acf5c291d714d69b88f6a0def9aeeaa => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:23.402218 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1d4a96a5-e4d0-409a-b8ed-da0fde1d8e27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:23.402520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:23.404028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:23.425702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a96a5-e4d0-409a-b8ed-da0fde1d8e27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:23.427237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a96a5-e4d0-409a-b8ed-da0fde1d8e27 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:23.443980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1086/2149] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:23 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:23.463809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a96a5-e4d0-409a-b8ed-da0fde1d8e27 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:23.469177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1d4a96a5-e4d0-409a-b8ed-da0fde1d8e27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:23.513630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a96a5-e4d0-409a-b8ed-da0fde1d8e27 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:23.613004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54212e3f-4ff7-4a82-9307-d5c3f8f59d0f admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:23.625678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1580241-b51a-4bb6-8dd1-bf6e1af55784 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:23.627365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1580241-b51a-4bb6-8dd1-bf6e1af55784 service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:23.629160 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1580241-b51a-4bb6-8dd1-bf6e1af55784 service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:23.630210 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1580241-b51a-4bb6-8dd1-bf6e1af55784 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:23.638744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:23.639596 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:23.639906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:23.640233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] PATH_INFO: `/v3/users/4705861fb1b4457cbc6f6f5d603bf939` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:23.644893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4705861fb1b4457cbc6f6f5d603bf939)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:23.645245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a1580241-b51a-4bb6-8dd1-bf6e1af55784 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:23.646917 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:23.648903 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:23.664918 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1580241-b51a-4bb6-8dd1-bf6e1af55784 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:23.668191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1580241-b51a-4bb6-8dd1-bf6e1af55784 service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:23.684270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1064/2150] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:23.732909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4705861fb1b4457cbc6f6f5d603bf939'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:23.747895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4705861fb1b4457cbc6f6f5d603bf939'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:23.749056 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4705861fb1b4457cbc6f6f5d603bf939'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:23.764418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4705861fb1b4457cbc6f6f5d603bf939'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:23.765302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4705861fb1b4457cbc6f6f5d603bf939'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:23.794766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1087/2151] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:23 2022] DELETE /identity/v3/users/4705861fb1b4457cbc6f6f5d603bf939 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:23.797538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeccc8d-ecee-42c5-bad0-540ad71232f1 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:23.797881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeccc8d-ecee-42c5-bad0-540ad71232f1 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:23.798212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeccc8d-ecee-42c5-bad0-540ad71232f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:23.827090 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-eeeccc8d-ecee-42c5-bad0-540ad71232f1 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:23.828332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-eeeccc8d-ecee-42c5-bad0-540ad71232f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:23.833522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:23.901027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:23.902499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:23.933719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1065/2152] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:23.945768 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeccc8d-ecee-42c5-bad0-540ad71232f1 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:23.947499 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-eeeccc8d-ecee-42c5-bad0-540ad71232f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:23.998479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f5ae1-0fea-42b8-a280-70cb0f3efdbb admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:24.004083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:24.005348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:24.005488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:24.005772 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] PATH_INFO: `/v3/users/b0f3576844744c538a4755221c4added` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:24.007044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f3576844744c538a4755221c4added)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:24.009094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:24.026981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeccc8d-ecee-42c5-bad0-540ad71232f1 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:24.037050 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafa4e4-96e7-41dc-8323-a1d830faa09c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:24.038008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafa4e4-96e7-41dc-8323-a1d830faa09c service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:24.038008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafa4e4-96e7-41dc-8323-a1d830faa09c service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:24.038214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafa4e4-96e7-41dc-8323-a1d830faa09c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:24.049229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafa4e4-96e7-41dc-8323-a1d830faa09c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:24.050749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafa4e4-96e7-41dc-8323-a1d830faa09c service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:24.060942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1066/2153] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:41:24.061788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafa4e4-96e7-41dc-8323-a1d830faa09c service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:24.063775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0fafa4e4-96e7-41dc-8323-a1d830faa09c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:24.077012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f3576844744c538a4755221c4added'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.082755 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafa4e4-96e7-41dc-8323-a1d830faa09c service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:24.088923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:24.089968 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:24.090318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:24.090634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] PATH_INFO: `/v3/projects/3fe0a286aca64c789b36e330bd231a99` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:24.092136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f3576844744c538a4755221c4added'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.092430 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fe0a286aca64c789b36e330bd231a99)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:24.093644 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f3576844744c538a4755221c4added'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.094631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:24.103947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f3576844744c538a4755221c4added'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.110837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f3576844744c538a4755221c4added'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.143941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:24.149565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1088/2154] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:23 2022] DELETE /identity/v3/users/b0f3576844744c538a4755221c4added => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:41:24.172592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fe0a286aca64c789b36e330bd231a99', 'request_id': 'req-07bf1619-edaf-472f-be44-1bc3f2b1cd98'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.194593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fe0a286aca64c789b36e330bd231a99', 'request_id': 'req-07bf1619-edaf-472f-be44-1bc3f2b1cd98'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.206592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fe0a286aca64c789b36e330bd231a99', 'request_id': 'req-07bf1619-edaf-472f-be44-1bc3f2b1cd98'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.223017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] The token cache is being invalidate because project 3fe0a286aca64c789b36e330bd231a99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:24.223276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.224195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.226653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1067/2155] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:24 2022] DELETE /identity/v3/projects/3fe0a286aca64c789b36e330bd231a99 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:24.230786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:24.232521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:24.344432 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ef045-cad1-4739-876a-18a58b8eb081 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:24.429673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:24.430713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:24.431129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:24.431510 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] PATH_INFO: `/v3/users/65aa58a021c04157a51406337000f475` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:24.432939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65aa58a021c04157a51406337000f475)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:24.434981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:24.446121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:24.448164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:24.488912 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf1619-edaf-472f-be44-1bc3f2b1cd98 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:24.494743 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50add6fd-958f-4214-85b2-4e385f7d0f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:24.495644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50add6fd-958f-4214-85b2-4e385f7d0f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:24.496041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50add6fd-958f-4214-85b2-4e385f7d0f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:24.496327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50add6fd-958f-4214-85b2-4e385f7d0f2f admin admin] PATH_INFO: `/v3/projects/6276528cc9f2420d9f400a986160ae37` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:24.498417 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50add6fd-958f-4214-85b2-4e385f7d0f2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6276528cc9f2420d9f400a986160ae37)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:24.509576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50add6fd-958f-4214-85b2-4e385f7d0f2f admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:24.525775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65aa58a021c04157a51406337000f475'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.534712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65aa58a021c04157a51406337000f475'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.535442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65aa58a021c04157a51406337000f475'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.562052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:24.567170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65aa58a021c04157a51406337000f475'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.569401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65aa58a021c04157a51406337000f475'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.589904 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-50add6fd-958f-4214-85b2-4e385f7d0f2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6276528cc9f2420d9f400a986160ae37', 'request_id': 'req-50add6fd-958f-4214-85b2-4e385f7d0f2f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.604471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-50add6fd-958f-4214-85b2-4e385f7d0f2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6276528cc9f2420d9f400a986160ae37', 'request_id': 'req-50add6fd-958f-4214-85b2-4e385f7d0f2f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.606152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1089/2156] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:24 2022] DELETE /identity/v3/users/65aa58a021c04157a51406337000f475 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:24.609264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:24.609264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:24.622531 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-50add6fd-958f-4214-85b2-4e385f7d0f2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6276528cc9f2420d9f400a986160ae37', 'request_id': 'req-50add6fd-958f-4214-85b2-4e385f7d0f2f'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.639804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-50add6fd-958f-4214-85b2-4e385f7d0f2f admin admin] The token cache is being invalidate because project 6276528cc9f2420d9f400a986160ae37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:24.640139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-50add6fd-958f-4214-85b2-4e385f7d0f2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.641471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-50add6fd-958f-4214-85b2-4e385f7d0f2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.643410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1068/2157] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:24 2022] DELETE /identity/v3/projects/6276528cc9f2420d9f400a986160ae37 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:41:24.648201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1030a-1a86-4461-8436-4f0189c4b9e2 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:24.648678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1030a-1a86-4461-8436-4f0189c4b9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:24.649087 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1030a-1a86-4461-8436-4f0189c4b9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:24.655582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:24.678771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-91b1030a-1a86-4461-8436-4f0189c4b9e2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:24.680796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-91b1030a-1a86-4461-8436-4f0189c4b9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:24.706518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5d807-cf04-44fb-9ba5-aafa867dcfc7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:24.724436 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19d862-3c12-41e5-86df-b0f1d88bb742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:24.725584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19d862-3c12-41e5-86df-b0f1d88bb742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:24.731714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19d862-3c12-41e5-86df-b0f1d88bb742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:24.731714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19d862-3c12-41e5-86df-b0f1d88bb742 admin admin] PATH_INFO: `/v3/projects/c24e32d2fb6e4a3db413acc06528d15e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:24.731714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19d862-3c12-41e5-86df-b0f1d88bb742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c24e32d2fb6e4a3db413acc06528d15e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:24.731714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19d862-3c12-41e5-86df-b0f1d88bb742 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:24.769224 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1069/2158] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:41:24.778573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75d09d-ab6f-41fa-ae66-04604ccf37ea None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:24.778863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75d09d-ab6f-41fa-ae66-04604ccf37ea None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:24.779149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75d09d-ab6f-41fa-ae66-04604ccf37ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:24.791069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6b19d862-3c12-41e5-86df-b0f1d88bb742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c24e32d2fb6e4a3db413acc06528d15e', 'request_id': 'req-6b19d862-3c12-41e5-86df-b0f1d88bb742'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.800525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6b19d862-3c12-41e5-86df-b0f1d88bb742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c24e32d2fb6e4a3db413acc06528d15e', 'request_id': 'req-6b19d862-3c12-41e5-86df-b0f1d88bb742'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.801395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-9b75d09d-ab6f-41fa-ae66-04604ccf37ea None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:24.802694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9b75d09d-ab6f-41fa-ae66-04604ccf37ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:24.808311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6b19d862-3c12-41e5-86df-b0f1d88bb742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c24e32d2fb6e4a3db413acc06528d15e', 'request_id': 'req-6b19d862-3c12-41e5-86df-b0f1d88bb742'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.820253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6b19d862-3c12-41e5-86df-b0f1d88bb742 admin admin] The token cache is being invalidate because project c24e32d2fb6e4a3db413acc06528d15e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:24.820951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6b19d862-3c12-41e5-86df-b0f1d88bb742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.821709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6b19d862-3c12-41e5-86df-b0f1d88bb742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.823617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1090/2159] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:24 2022] DELETE /identity/v3/projects/c24e32d2fb6e4a3db413acc06528d15e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:24.829465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa31886-96ad-446a-be18-a776a799293c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:24.829465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa31886-96ad-446a-be18-a776a799293c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:24.830712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa31886-96ad-446a-be18-a776a799293c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:24.850560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3fa31886-96ad-446a-be18-a776a799293c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:24.851930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3fa31886-96ad-446a-be18-a776a799293c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:24.897097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1070/2160] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:24.898094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75d09d-ab6f-41fa-ae66-04604ccf37ea None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:24.899846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9b75d09d-ab6f-41fa-ae66-04604ccf37ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:24.911678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75d09d-ab6f-41fa-ae66-04604ccf37ea None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:24.916561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:24.917496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:24.917883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:24.918286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] PATH_INFO: `/v3/projects/5edca2f0196a4c67a57aaab43a1a7825` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:24.920221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5edca2f0196a4c67a57aaab43a1a7825)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:24.922259 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:24.925154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1091/2161] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:24.926838 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa31886-96ad-446a-be18-a776a799293c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:24.938084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa31886-96ad-446a-be18-a776a799293c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:24.944225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424ddf0-adc1-45e3-9253-c6cfd313ade1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:24.944890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424ddf0-adc1-45e3-9253-c6cfd313ade1 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:24.945171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424ddf0-adc1-45e3-9253-c6cfd313ade1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:24.945448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424ddf0-adc1-45e3-9253-c6cfd313ade1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:24.946210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424ddf0-adc1-45e3-9253-c6cfd313ade1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:24.946528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424ddf0-adc1-45e3-9253-c6cfd313ade1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:24.948112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424ddf0-adc1-45e3-9253-c6cfd313ade1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:24.965325 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1092/2162] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:24.966987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70b56a-ae64-4a6a-a246-771e098882a6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:24.967274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70b56a-ae64-4a6a-a246-771e098882a6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:24.967550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70b56a-ae64-4a6a-a246-771e098882a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:24.987097 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5edca2f0196a4c67a57aaab43a1a7825', 'request_id': 'req-e01024fd-40da-43c9-96af-502ca4ae67c7'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.989389 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5d70b56a-ae64-4a6a-a246-771e098882a6 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:24.990617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5d70b56a-ae64-4a6a-a246-771e098882a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:24.993622 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5edca2f0196a4c67a57aaab43a1a7825', 'request_id': 'req-e01024fd-40da-43c9-96af-502ca4ae67c7'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:24.995002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:25.001601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5edca2f0196a4c67a57aaab43a1a7825', 'request_id': 'req-e01024fd-40da-43c9-96af-502ca4ae67c7'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:25.022379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] The token cache is being invalidate because project 5edca2f0196a4c67a57aaab43a1a7825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:25.022742 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:25.023828 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:25.025727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1071/2163] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:24 2022] DELETE /identity/v3/projects/5edca2f0196a4c67a57aaab43a1a7825 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:25.030116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.030116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:25.034575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:25.058855 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1093/2164] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:25.060775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54b96e-ee3c-4c17-9bb2-6729a875f7dc None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.061076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54b96e-ee3c-4c17-9bb2-6729a875f7dc None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.061351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54b96e-ee3c-4c17-9bb2-6729a875f7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.081603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6f54b96e-ee3c-4c17-9bb2-6729a875f7dc None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:25.082970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6f54b96e-ee3c-4c17-9bb2-6729a875f7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:25.092001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1094/2165] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:25.092726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54b96e-ee3c-4c17-9bb2-6729a875f7dc None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.097724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01024fd-40da-43c9-96af-502ca4ae67c7 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.102931 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fa5c1-e7e0-47b2-909f-66ba4809e00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.103361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54b96e-ee3c-4c17-9bb2-6729a875f7dc None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.103596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fa5c1-e7e0-47b2-909f-66ba4809e00c admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.103886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fa5c1-e7e0-47b2-909f-66ba4809e00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.104162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fa5c1-e7e0-47b2-909f-66ba4809e00c admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.104892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fa5c1-e7e0-47b2-909f-66ba4809e00c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.107388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fa5c1-e7e0-47b2-909f-66ba4809e00c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.110282 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c6366-1b99-4372-904c-15c821958284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.111041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c6366-1b99-4372-904c-15c821958284 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.111395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c6366-1b99-4372-904c-15c821958284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.111743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c6366-1b99-4372-904c-15c821958284 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.112557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c6366-1b99-4372-904c-15c821958284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:25.112939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c6366-1b99-4372-904c-15c821958284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.114764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c6366-1b99-4372-904c-15c821958284 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.127697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1095/2166] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:25.128993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf702e9-8966-4595-8608-45cd71bd275c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.129260 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf702e9-8966-4595-8608-45cd71bd275c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.129533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf702e9-8966-4595-8608-45cd71bd275c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.144022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1072/2167] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:25 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:25.144022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fa5c1-e7e0-47b2-909f-66ba4809e00c admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.149689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7bf702e9-8966-4595-8608-45cd71bd275c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:25.151081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7bf702e9-8966-4595-8608-45cd71bd275c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:25.212869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fa5c1-e7e0-47b2-909f-66ba4809e00c admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.213902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1096/2168] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:25.218538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf702e9-8966-4595-8608-45cd71bd275c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.218688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447c004-0a4f-4c46-a7ac-bc986f05cb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.219568 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447c004-0a4f-4c46-a7ac-bc986f05cb95 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.219872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447c004-0a4f-4c46-a7ac-bc986f05cb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.220242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447c004-0a4f-4c46-a7ac-bc986f05cb95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.221055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447c004-0a4f-4c46-a7ac-bc986f05cb95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.223491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447c004-0a4f-4c46-a7ac-bc986f05cb95 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.236264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf702e9-8966-4595-8608-45cd71bd275c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.241372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adc289-c549-4d72-8e61-f094dcbb6924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.242120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adc289-c549-4d72-8e61-f094dcbb6924 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.242411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adc289-c549-4d72-8e61-f094dcbb6924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.242685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adc289-c549-4d72-8e61-f094dcbb6924 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.243424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79adc289-c549-4d72-8e61-f094dcbb6924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.245390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79adc289-c549-4d72-8e61-f094dcbb6924 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.252862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1073/2169] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:25 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:25.257813 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447c004-0a4f-4c46-a7ac-bc986f05cb95 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.271286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1097/2170] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:25 2022] POST /identity/v3/users => generated 288 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:25.271286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adc289-c549-4d72-8e61-f094dcbb6924 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.280023 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adc289-c549-4d72-8e61-f094dcbb6924 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.317392 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:25.334244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447c004-0a4f-4c46-a7ac-bc986f05cb95 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.344557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e91d6c-160f-4cab-a4c2-347536dd77ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.345268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e91d6c-160f-4cab-a4c2-347536dd77ad admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.345553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e91d6c-160f-4cab-a4c2-347536dd77ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.345831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e91d6c-160f-4cab-a4c2-347536dd77ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.346613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e91d6c-160f-4cab-a4c2-347536dd77ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:25.346923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e91d6c-160f-4cab-a4c2-347536dd77ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.348535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e91d6c-160f-4cab-a4c2-347536dd77ad admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.351741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae848a6-5924-4fcc-8c81-c3c261389b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.352487 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae848a6-5924-4fcc-8c81-c3c261389b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.352863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae848a6-5924-4fcc-8c81-c3c261389b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.353188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae848a6-5924-4fcc-8c81-c3c261389b7b admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.354187 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae848a6-5924-4fcc-8c81-c3c261389b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.356172 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae848a6-5924-4fcc-8c81-c3c261389b7b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.365079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1074/2171] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:25.365079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e91d6c-160f-4cab-a4c2-347536dd77ad admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.373493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e91d6c-160f-4cab-a4c2-347536dd77ad admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.378338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cda99a-cf8a-416e-bc1b-e7386893df07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.378542 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1098/2172] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:25 2022] POST /identity/v3/users => generated 302 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:25.378985 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cda99a-cf8a-416e-bc1b-e7386893df07 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.379391 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cda99a-cf8a-416e-bc1b-e7386893df07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.379484 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae848a6-5924-4fcc-8c81-c3c261389b7b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.379672 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cda99a-cf8a-416e-bc1b-e7386893df07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.380401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cda99a-cf8a-416e-bc1b-e7386893df07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.382210 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cda99a-cf8a-416e-bc1b-e7386893df07 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.391849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae848a6-5924-4fcc-8c81-c3c261389b7b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.393634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1075/2173] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:25.394636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cda99a-cf8a-416e-bc1b-e7386893df07 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.397844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d32c3d-26d4-4549-b697-e620f8a745dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.398991 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d32c3d-26d4-4549-b697-e620f8a745dc admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.399277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d32c3d-26d4-4549-b697-e620f8a745dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.399550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d32c3d-26d4-4549-b697-e620f8a745dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.400247 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d32c3d-26d4-4549-b697-e620f8a745dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.402201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d32c3d-26d4-4549-b697-e620f8a745dc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.411898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cda99a-cf8a-416e-bc1b-e7386893df07 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.417212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30aea7-e159-49ea-8d77-36bcc02f3a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.417891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30aea7-e159-49ea-8d77-36bcc02f3a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.418210 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30aea7-e159-49ea-8d77-36bcc02f3a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.418529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30aea7-e159-49ea-8d77-36bcc02f3a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.419895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30aea7-e159-49ea-8d77-36bcc02f3a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.421881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30aea7-e159-49ea-8d77-36bcc02f3a12 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.435490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1076/2174] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:25.436762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30aea7-e159-49ea-8d77-36bcc02f3a12 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.440066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1099/2175] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:25 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:25.440966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d32c3d-26d4-4549-b697-e620f8a745dc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.448668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30aea7-e159-49ea-8d77-36bcc02f3a12 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.462961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:25.516427 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d32c3d-26d4-4549-b697-e620f8a745dc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.521576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78791a15-d718-45cd-965e-cc8a18acecea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.522293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78791a15-d718-45cd-965e-cc8a18acecea admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.526142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78791a15-d718-45cd-965e-cc8a18acecea admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.526142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78791a15-d718-45cd-965e-cc8a18acecea admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.526142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78791a15-d718-45cd-965e-cc8a18acecea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.526142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78791a15-d718-45cd-965e-cc8a18acecea admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.540024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1100/2176] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:25.540024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78791a15-d718-45cd-965e-cc8a18acecea admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.543503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fcd99-887a-4c46-9087-5599c2b6bebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.544873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fcd99-887a-4c46-9087-5599c2b6bebf admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.545820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fcd99-887a-4c46-9087-5599c2b6bebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.545820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fcd99-887a-4c46-9087-5599c2b6bebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.546342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fcd99-887a-4c46-9087-5599c2b6bebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.547937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fcd99-887a-4c46-9087-5599c2b6bebf admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.552159 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78791a15-d718-45cd-965e-cc8a18acecea admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.557702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb171-2892-413d-a8cd-64c84f235486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.561322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb171-2892-413d-a8cd-64c84f235486 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.561322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb171-2892-413d-a8cd-64c84f235486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.561322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb171-2892-413d-a8cd-64c84f235486 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.561880 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbb171-2892-413d-a8cd-64c84f235486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.563030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1077/2177] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:25.563949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fcd99-887a-4c46-9087-5599c2b6bebf admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.564431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbb171-2892-413d-a8cd-64c84f235486 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.574102 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fcd99-887a-4c46-9087-5599c2b6bebf admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.579846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884cca7-c8a0-406c-98aa-fe1574f2c625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.580818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884cca7-c8a0-406c-98aa-fe1574f2c625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.581110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884cca7-c8a0-406c-98aa-fe1574f2c625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.581383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884cca7-c8a0-406c-98aa-fe1574f2c625 admin admin] PATH_INFO: `/v3/projects/499aea7bcd1f4ef9a840e6ffdaec46d2/users/f01542a96186429591c27d6f03d9116e/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.597948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1101/2178] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:25 2022] POST /identity/v3/users => generated 284 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:25.597948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb171-2892-413d-a8cd-64c84f235486 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.608525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb171-2892-413d-a8cd-64c84f235486 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.611436 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884cca7-c8a0-406c-98aa-fe1574f2c625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=499aea7bcd1f4ef9a840e6ffdaec46d2, user_id=f01542a96186429591c27d6f03d9116e, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.613795 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b96f1c-48ca-4f96-ba7e-a60a956e6fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.614291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884cca7-c8a0-406c-98aa-fe1574f2c625 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.614719 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b96f1c-48ca-4f96-ba7e-a60a956e6fff admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.614980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b96f1c-48ca-4f96-ba7e-a60a956e6fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.615258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b96f1c-48ca-4f96-ba7e-a60a956e6fff admin admin] PATH_INFO: `/v3/projects/ddbe4a761eef4f7da9c514dff339f936/users/308de2545b7d49698fefb004057a9dcb/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.628705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1078/2179] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:25 2022] PUT /identity/v3/projects/499aea7bcd1f4ef9a840e6ffdaec46d2/users/f01542a96186429591c27d6f03d9116e/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:25.629441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884cca7-c8a0-406c-98aa-fe1574f2c625 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.633100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b96f1c-48ca-4f96-ba7e-a60a956e6fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbe4a761eef4f7da9c514dff339f936, user_id=308de2545b7d49698fefb004057a9dcb, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.638500 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b96f1c-48ca-4f96-ba7e-a60a956e6fff admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.656683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1102/2180] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:25 2022] PUT /identity/v3/projects/ddbe4a761eef4f7da9c514dff339f936/users/308de2545b7d49698fefb004057a9dcb/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:25.657817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6e422-49b6-4f1f-9fa7-09081638d2f4 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.658152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6e422-49b6-4f1f-9fa7-09081638d2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.658605 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6e422-49b6-4f1f-9fa7-09081638d2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.691865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-45f6e422-49b6-4f1f-9fa7-09081638d2f4 None None] MFA Rules not processed for user `f01542a96186429591c27d6f03d9116e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:25.693074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-45f6e422-49b6-4f1f-9fa7-09081638d2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:25.711443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884cca7-c8a0-406c-98aa-fe1574f2c625 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.761187 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:25.764057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1103/2181] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:25.765334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab452b6b-9091-40ff-b6e4-36f0537a1b1d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.765625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab452b6b-9091-40ff-b6e4-36f0537a1b1d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.765904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab452b6b-9091-40ff-b6e4-36f0537a1b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.783126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcb204-1ca8-45ea-904c-a35a0cb9dcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.783811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcb204-1ca8-45ea-904c-a35a0cb9dcf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.784093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcb204-1ca8-45ea-904c-a35a0cb9dcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.784370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcb204-1ca8-45ea-904c-a35a0cb9dcf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.785142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcb204-1ca8-45ea-904c-a35a0cb9dcf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.786790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcb204-1ca8-45ea-904c-a35a0cb9dcf0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.797349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1079/2182] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:25.798318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcb204-1ca8-45ea-904c-a35a0cb9dcf0 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.799990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ab452b6b-9091-40ff-b6e4-36f0537a1b1d None None] MFA Rules not processed for user `308de2545b7d49698fefb004057a9dcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:25.801244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ab452b6b-9091-40ff-b6e4-36f0537a1b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:25.808824 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcb204-1ca8-45ea-904c-a35a0cb9dcf0 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.813681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f2ea4-3b2f-4c90-9472-6cb9c11a7b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.814382 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f2ea4-3b2f-4c90-9472-6cb9c11a7b20 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.814670 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f2ea4-3b2f-4c90-9472-6cb9c11a7b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.814951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f2ea4-3b2f-4c90-9472-6cb9c11a7b20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.815753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f2ea4-3b2f-4c90-9472-6cb9c11a7b20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.817669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f2ea4-3b2f-4c90-9472-6cb9c11a7b20 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.847667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:25.856708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1080/2183] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:25 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:25.856708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f2ea4-3b2f-4c90-9472-6cb9c11a7b20 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.871484 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1104/2184] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:25.874019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab452b6b-9091-40ff-b6e4-36f0537a1b1d None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.885394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab452b6b-9091-40ff-b6e4-36f0537a1b1d None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.925770 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f2ea4-3b2f-4c90-9472-6cb9c11a7b20 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.931106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485f86b-3b35-437e-9d73-d222ad58fe0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.932044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485f86b-3b35-437e-9d73-d222ad58fe0e admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.932436 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485f86b-3b35-437e-9d73-d222ad58fe0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.932844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485f86b-3b35-437e-9d73-d222ad58fe0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.933688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485f86b-3b35-437e-9d73-d222ad58fe0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.935492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485f86b-3b35-437e-9d73-d222ad58fe0e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.943584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa616e91-374b-4c70-8e08-90fa4ca6aa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.944274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa616e91-374b-4c70-8e08-90fa4ca6aa71 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.944564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa616e91-374b-4c70-8e08-90fa4ca6aa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.944846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa616e91-374b-4c70-8e08-90fa4ca6aa71 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:25.945552 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa616e91-374b-4c70-8e08-90fa4ca6aa71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:25.945690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1081/2185] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:25.947706 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa616e91-374b-4c70-8e08-90fa4ca6aa71 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:25.949588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485f86b-3b35-437e-9d73-d222ad58fe0e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.951821 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0485f86b-3b35-437e-9d73-d222ad58fe0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:25.974859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1105/2186] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:25 2022] POST /identity/v3/users => generated 309 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:25.974859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa616e91-374b-4c70-8e08-90fa4ca6aa71 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:25.982170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa616e91-374b-4c70-8e08-90fa4ca6aa71 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:25.987178 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691060a-e43b-441a-ad70-0c3d344c5a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:25.988180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691060a-e43b-441a-ad70-0c3d344c5a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:25.988610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691060a-e43b-441a-ad70-0c3d344c5a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:25.988949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691060a-e43b-441a-ad70-0c3d344c5a09 admin admin] PATH_INFO: `/v3/projects/e97970687da94cd8a4aef929b07bf2ff/users/319718e05ce84d16953162fcaacf1d8f/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:26.005344 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691060a-e43b-441a-ad70-0c3d344c5a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e97970687da94cd8a4aef929b07bf2ff, user_id=319718e05ce84d16953162fcaacf1d8f, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:26.008093 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691060a-e43b-441a-ad70-0c3d344c5a09 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:26.020223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485f86b-3b35-437e-9d73-d222ad58fe0e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:26.023904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1106/2187] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:25 2022] PUT /identity/v3/projects/e97970687da94cd8a4aef929b07bf2ff/users/319718e05ce84d16953162fcaacf1d8f/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:26.024888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691060a-e43b-441a-ad70-0c3d344c5a09 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:26.080095 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf9160-456a-4e43-a062-78ee8acfcbb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:26.080800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf9160-456a-4e43-a062-78ee8acfcbb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:26.081084 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf9160-456a-4e43-a062-78ee8acfcbb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:26.081362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf9160-456a-4e43-a062-78ee8acfcbb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:26.106460 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691060a-e43b-441a-ad70-0c3d344c5a09 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:26.108233 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:26.113001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fee6a6-7001-40bd-960d-9898552ae746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:26.113678 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fee6a6-7001-40bd-960d-9898552ae746 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:26.113995 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fee6a6-7001-40bd-960d-9898552ae746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:26.114278 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fee6a6-7001-40bd-960d-9898552ae746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:26.115071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fee6a6-7001-40bd-960d-9898552ae746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:26.116635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fee6a6-7001-40bd-960d-9898552ae746 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:26.130832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1107/2188] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:26.130832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed809b1-5e10-47e4-9ce8-c9bf15127c2c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:26.130832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed809b1-5e10-47e4-9ce8-c9bf15127c2c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:26.130832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed809b1-5e10-47e4-9ce8-c9bf15127c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:26.143595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf9160-456a-4e43-a062-78ee8acfcbb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:26.145138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf9160-456a-4e43-a062-78ee8acfcbb5 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:26.155796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1082/2189] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:26.156756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf9160-456a-4e43-a062-78ee8acfcbb5 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:26.162618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-eed809b1-5e10-47e4-9ce8-c9bf15127c2c None None] MFA Rules not processed for user `319718e05ce84d16953162fcaacf1d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:26.164426 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-eed809b1-5e10-47e4-9ce8-c9bf15127c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:26.167240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf9160-456a-4e43-a062-78ee8acfcbb5 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:26.172016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650b7b4-aa49-4885-95a1-1804495d7e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:26.172919 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650b7b4-aa49-4885-95a1-1804495d7e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:26.173205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650b7b4-aa49-4885-95a1-1804495d7e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:26.173487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650b7b4-aa49-4885-95a1-1804495d7e7b admin admin] PATH_INFO: `/v3/projects/d655165ef3ff456aa9d210a3c10061e6/users/12e2110eadaa4b2b8d31978bc47a0ae8/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:26.195559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650b7b4-aa49-4885-95a1-1804495d7e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d655165ef3ff456aa9d210a3c10061e6, user_id=12e2110eadaa4b2b8d31978bc47a0ae8, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:26.201309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650b7b4-aa49-4885-95a1-1804495d7e7b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:26.215379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1083/2190] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:26 2022] PUT /identity/v3/projects/d655165ef3ff456aa9d210a3c10061e6/users/12e2110eadaa4b2b8d31978bc47a0ae8/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:26.219396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650b7b4-aa49-4885-95a1-1804495d7e7b admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:26.227194 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:26.260374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1108/2191] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:26.300361 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650b7b4-aa49-4885-95a1-1804495d7e7b admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:26.305205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a14975-bb4b-40e8-8f9c-048bffbd018c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:26.305874 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a14975-bb4b-40e8-8f9c-048bffbd018c admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:26.306200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a14975-bb4b-40e8-8f9c-048bffbd018c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:26.306490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a14975-bb4b-40e8-8f9c-048bffbd018c admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:26.307218 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a14975-bb4b-40e8-8f9c-048bffbd018c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:26.309034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a14975-bb4b-40e8-8f9c-048bffbd018c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:26.321373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1084/2192] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:41:26.326865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed809b1-5e10-47e4-9ce8-c9bf15127c2c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:26.339985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed809b1-5e10-47e4-9ce8-c9bf15127c2c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:26.344649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8493c52-3b70-4d2c-bcd7-46e28ef1c60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:26.345562 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8493c52-3b70-4d2c-bcd7-46e28ef1c60d admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:26.345862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8493c52-3b70-4d2c-bcd7-46e28ef1c60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:26.346273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8493c52-3b70-4d2c-bcd7-46e28ef1c60d admin admin] PATH_INFO: `/v3/projects/d655165ef3ff456aa9d210a3c10061e6/users/12e2110eadaa4b2b8d31978bc47a0ae8/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:26.348276 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8493c52-3b70-4d2c-bcd7-46e28ef1c60d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d655165ef3ff456aa9d210a3c10061e6, user_id=12e2110eadaa4b2b8d31978bc47a0ae8, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:26.350993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8493c52-3b70-4d2c-bcd7-46e28ef1c60d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:26.364442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1109/2193] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:26 2022] PUT /identity/v3/projects/d655165ef3ff456aa9d210a3c10061e6/users/12e2110eadaa4b2b8d31978bc47a0ae8/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:26.377257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61785502-1de3-4835-9548-06e4634a72ce None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:26.377757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61785502-1de3-4835-9548-06e4634a72ce None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:26.378969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61785502-1de3-4835-9548-06e4634a72ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:26.430178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-61785502-1de3-4835-9548-06e4634a72ce None None] MFA Rules not processed for user `12e2110eadaa4b2b8d31978bc47a0ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:26.431945 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-61785502-1de3-4835-9548-06e4634a72ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:26.521089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1085/2194] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:26 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:26.531773 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8493c52-3b70-4d2c-bcd7-46e28ef1c60d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:26.592562 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8493c52-3b70-4d2c-bcd7-46e28ef1c60d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:26.596230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfa17a-e3a4-4c5f-8927-893535df1226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:26.596877 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfa17a-e3a4-4c5f-8927-893535df1226 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:26.597164 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfa17a-e3a4-4c5f-8927-893535df1226 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:26.597443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfa17a-e3a4-4c5f-8927-893535df1226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:26.607658 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfa17a-e3a4-4c5f-8927-893535df1226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:26.609101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfa17a-e3a4-4c5f-8927-893535df1226 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:26.619250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1110/2195] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:26 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:26.847427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61785502-1de3-4835-9548-06e4634a72ce None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:26.878176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:26.917526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61785502-1de3-4835-9548-06e4634a72ce None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:26.922724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:26.923594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:26.923873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:26.924144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] PATH_INFO: `/v3/users/f01542a96186429591c27d6f03d9116e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:26.925349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=f01542a96186429591c27d6f03d9116e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:26.927324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:26.973634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f01542a96186429591c27d6f03d9116e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:26.974201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f01542a96186429591c27d6f03d9116e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:26.974807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f01542a96186429591c27d6f03d9116e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:26.982479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f01542a96186429591c27d6f03d9116e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:26.991516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f01542a96186429591c27d6f03d9116e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:27.008836 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1086/2196] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:26 2022] DELETE /identity/v3/users/f01542a96186429591c27d6f03d9116e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:27.013440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c475d7b-acf5-4def-98db-bc445c9b0e6c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:27.013734 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c475d7b-acf5-4def-98db-bc445c9b0e6c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:27.014048 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c475d7b-acf5-4def-98db-bc445c9b0e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:27.033404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0c475d7b-acf5-4def-98db-bc445c9b0e6c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:27.034789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0c475d7b-acf5-4def-98db-bc445c9b0e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:27.095033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1111/2197] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:27.104844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:27.106336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:27.165434 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea108d4-a78b-458c-936d-bfebe62517ca admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:27.172258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccb1d6-e3ae-41a5-8809-a52b234a6d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:27.172940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccb1d6-e3ae-41a5-8809-a52b234a6d49 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:27.173222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccb1d6-e3ae-41a5-8809-a52b234a6d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:27.173496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccb1d6-e3ae-41a5-8809-a52b234a6d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:27.183233 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccb1d6-e3ae-41a5-8809-a52b234a6d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:27.185602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccb1d6-e3ae-41a5-8809-a52b234a6d49 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:27.195505 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1087/2198] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:41:27.476022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c475d7b-acf5-4def-98db-bc445c9b0e6c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:27.477717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0c475d7b-acf5-4def-98db-bc445c9b0e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:27.490863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c475d7b-acf5-4def-98db-bc445c9b0e6c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:27.495627 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:27.496595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:27.496777 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:27.497564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] PATH_INFO: `/v3/projects/499aea7bcd1f4ef9a840e6ffdaec46d2` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:27.498799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] RBAC: Authorizing `identity:delete_project(project_id=499aea7bcd1f4ef9a840e6ffdaec46d2)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:27.501167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:27.552201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '499aea7bcd1f4ef9a840e6ffdaec46d2', 'request_id': 'req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:27.560106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '499aea7bcd1f4ef9a840e6ffdaec46d2', 'request_id': 'req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:27.568424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '499aea7bcd1f4ef9a840e6ffdaec46d2', 'request_id': 'req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:27.570637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:27.581641 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] The token cache is being invalidate because project 499aea7bcd1f4ef9a840e6ffdaec46d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:27.582083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:27.582938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:27.584579 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1112/2199] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:27 2022] DELETE /identity/v3/projects/499aea7bcd1f4ef9a840e6ffdaec46d2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:27.588749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccb1d6-e3ae-41a5-8809-a52b234a6d49 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:27.599898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccb1d6-e3ae-41a5-8809-a52b234a6d49 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:27.650898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:27.669059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901ad74-13f3-4ee7-baee-24e8ab137853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:27.669972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901ad74-13f3-4ee7-baee-24e8ab137853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:27.670260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901ad74-13f3-4ee7-baee-24e8ab137853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:27.670568 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901ad74-13f3-4ee7-baee-24e8ab137853 admin admin] PATH_INFO: `/v3/users/12e2110eadaa4b2b8d31978bc47a0ae8` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:27.671779 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901ad74-13f3-4ee7-baee-24e8ab137853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e2110eadaa4b2b8d31978bc47a0ae8)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:27.673706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901ad74-13f3-4ee7-baee-24e8ab137853 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:27.726323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5901ad74-13f3-4ee7-baee-24e8ab137853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e2110eadaa4b2b8d31978bc47a0ae8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:27.726974 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5901ad74-13f3-4ee7-baee-24e8ab137853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e2110eadaa4b2b8d31978bc47a0ae8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:27.727579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5901ad74-13f3-4ee7-baee-24e8ab137853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e2110eadaa4b2b8d31978bc47a0ae8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:27.741134 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5901ad74-13f3-4ee7-baee-24e8ab137853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e2110eadaa4b2b8d31978bc47a0ae8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:27.759964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5901ad74-13f3-4ee7-baee-24e8ab137853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e2110eadaa4b2b8d31978bc47a0ae8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:27.791163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1088/2200] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:27 2022] DELETE /identity/v3/users/12e2110eadaa4b2b8d31978bc47a0ae8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:28.103884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:28.106116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:28.128857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a850d-a6b9-4c35-8edd-9e2b9cd8623b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:28.202919 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524e230-b5f8-4706-b79b-3bcca3971ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:28.203785 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524e230-b5f8-4706-b79b-3bcca3971ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:28.204065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524e230-b5f8-4706-b79b-3bcca3971ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:28.204336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524e230-b5f8-4706-b79b-3bcca3971ed0 admin admin] PATH_INFO: `/v3/projects/d655165ef3ff456aa9d210a3c10061e6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:28.205638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524e230-b5f8-4706-b79b-3bcca3971ed0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d655165ef3ff456aa9d210a3c10061e6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:28.207672 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524e230-b5f8-4706-b79b-3bcca3971ed0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:28.260696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1524e230-b5f8-4706-b79b-3bcca3971ed0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd655165ef3ff456aa9d210a3c10061e6', 'request_id': 'req-1524e230-b5f8-4706-b79b-3bcca3971ed0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:28.268656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1524e230-b5f8-4706-b79b-3bcca3971ed0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd655165ef3ff456aa9d210a3c10061e6', 'request_id': 'req-1524e230-b5f8-4706-b79b-3bcca3971ed0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:28.278028 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1524e230-b5f8-4706-b79b-3bcca3971ed0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd655165ef3ff456aa9d210a3c10061e6', 'request_id': 'req-1524e230-b5f8-4706-b79b-3bcca3971ed0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:28.288705 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1524e230-b5f8-4706-b79b-3bcca3971ed0 admin admin] The token cache is being invalidate because project d655165ef3ff456aa9d210a3c10061e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:28.288988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1524e230-b5f8-4706-b79b-3bcca3971ed0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:28.289623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1524e230-b5f8-4706-b79b-3bcca3971ed0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:28.291159 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1113/2201] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:28 2022] DELETE /identity/v3/projects/d655165ef3ff456aa9d210a3c10061e6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:28.296809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e215ce-54f7-4304-aedb-29bd5a6660fb None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:28.297280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e215ce-54f7-4304-aedb-29bd5a6660fb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:28.297358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e215ce-54f7-4304-aedb-29bd5a6660fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:28.316558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-03e215ce-54f7-4304-aedb-29bd5a6660fb None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:28.317772 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-03e215ce-54f7-4304-aedb-29bd5a6660fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:28.378190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1089/2202] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:28.385796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a7d27-f32d-4044-a7f9-104571581d32 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:28.386113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a7d27-f32d-4044-a7f9-104571581d32 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:28.386390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a7d27-f32d-4044-a7f9-104571581d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:28.407765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-721a7d27-f32d-4044-a7f9-104571581d32 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:28.409004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-721a7d27-f32d-4044-a7f9-104571581d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:28.417016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1114/2203] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:28.422327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e215ce-54f7-4304-aedb-29bd5a6660fb None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:28.431945 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e215ce-54f7-4304-aedb-29bd5a6660fb None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:28.436596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f8cc2-a886-4bbf-9bca-ddfb5da73987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:28.437265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f8cc2-a886-4bbf-9bca-ddfb5da73987 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:28.437548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f8cc2-a886-4bbf-9bca-ddfb5da73987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:28.437824 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f8cc2-a886-4bbf-9bca-ddfb5da73987 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:28.438585 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f8cc2-a886-4bbf-9bca-ddfb5da73987 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:28.438925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f8cc2-a886-4bbf-9bca-ddfb5da73987 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:28.440520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f8cc2-a886-4bbf-9bca-ddfb5da73987 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:28.452048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1090/2204] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:28.456539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a7d27-f32d-4044-a7f9-104571581d32 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:28.466514 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a7d27-f32d-4044-a7f9-104571581d32 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:28.471223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e655ce-1894-4a8b-8ad7-323de5a2d06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:28.473279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e655ce-1894-4a8b-8ad7-323de5a2d06c admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:28.473589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e655ce-1894-4a8b-8ad7-323de5a2d06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:28.473869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e655ce-1894-4a8b-8ad7-323de5a2d06c admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:28.474658 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e655ce-1894-4a8b-8ad7-323de5a2d06c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:28.476749 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e655ce-1894-4a8b-8ad7-323de5a2d06c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:28.511237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1115/2205] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:28 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:28.515494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f8cc2-a886-4bbf-9bca-ddfb5da73987 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:28.532088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f8cc2-a886-4bbf-9bca-ddfb5da73987 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:28.576201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:28.588082 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf85d4-f216-4d9c-8ec1-b16aad4cf9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:28.588750 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf85d4-f216-4d9c-8ec1-b16aad4cf9d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:28.589033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf85d4-f216-4d9c-8ec1-b16aad4cf9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:28.589309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf85d4-f216-4d9c-8ec1-b16aad4cf9d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:28.590066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf85d4-f216-4d9c-8ec1-b16aad4cf9d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:28.592470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf85d4-f216-4d9c-8ec1-b16aad4cf9d5 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:28.613264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1091/2206] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:28 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:41:28.617438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e655ce-1894-4a8b-8ad7-323de5a2d06c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:28.628004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e655ce-1894-4a8b-8ad7-323de5a2d06c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:28.632619 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d245d8f-2192-4704-951c-32192013e5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:28.633272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d245d8f-2192-4704-951c-32192013e5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:28.633551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d245d8f-2192-4704-951c-32192013e5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:28.633823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d245d8f-2192-4704-951c-32192013e5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:28.634579 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d245d8f-2192-4704-951c-32192013e5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:28.636173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d245d8f-2192-4704-951c-32192013e5d4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:28.647127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1116/2207] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:28.651886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf85d4-f216-4d9c-8ec1-b16aad4cf9d5 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:28.662750 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf85d4-f216-4d9c-8ec1-b16aad4cf9d5 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:28.667363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93e01b-49dc-40ee-8ffb-6ab035d1c099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:28.668020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93e01b-49dc-40ee-8ffb-6ab035d1c099 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:28.668318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93e01b-49dc-40ee-8ffb-6ab035d1c099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:28.668926 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93e01b-49dc-40ee-8ffb-6ab035d1c099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:28.670582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93e01b-49dc-40ee-8ffb-6ab035d1c099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:28.673570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93e01b-49dc-40ee-8ffb-6ab035d1c099 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:28.685372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1092/2208] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:28.689715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d245d8f-2192-4704-951c-32192013e5d4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:28.700800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d245d8f-2192-4704-951c-32192013e5d4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:28.705456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a498ef-15a7-4819-851e-b0daba8e39a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:28.706371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a498ef-15a7-4819-851e-b0daba8e39a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:28.706679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a498ef-15a7-4819-851e-b0daba8e39a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:28.706953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a498ef-15a7-4819-851e-b0daba8e39a1 admin admin] PATH_INFO: `/v3/projects/10c02ecd38ca45279ab37bf17cc78c82/users/3c29e22e7f4a4e94a5200ea511dc5590/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:28.721592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a498ef-15a7-4819-851e-b0daba8e39a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c02ecd38ca45279ab37bf17cc78c82, user_id=3c29e22e7f4a4e94a5200ea511dc5590, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:28.724277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a498ef-15a7-4819-851e-b0daba8e39a1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:28.738243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1117/2209] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:28 2022] PUT /identity/v3/projects/10c02ecd38ca45279ab37bf17cc78c82/users/3c29e22e7f4a4e94a5200ea511dc5590/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:41:28.749684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5d972-5d93-48f7-8097-12a4391bf316 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:28.749997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5d972-5d93-48f7-8097-12a4391bf316 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:28.750273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5d972-5d93-48f7-8097-12a4391bf316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:28.789235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-bec5d972-5d93-48f7-8097-12a4391bf316 None None] MFA Rules not processed for user `3c29e22e7f4a4e94a5200ea511dc5590`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:28.790514 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-bec5d972-5d93-48f7-8097-12a4391bf316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:28.857380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1093/2210] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:28.862442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a498ef-15a7-4819-851e-b0daba8e39a1 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:28.884589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:28.930175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a498ef-15a7-4819-851e-b0daba8e39a1 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:28.934859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706fd18-bd63-4dee-a10f-cda566e0fd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:28.935656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706fd18-bd63-4dee-a10f-cda566e0fd06 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:28.939587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706fd18-bd63-4dee-a10f-cda566e0fd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:28.939892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706fd18-bd63-4dee-a10f-cda566e0fd06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:28.940612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706fd18-bd63-4dee-a10f-cda566e0fd06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:28.943448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706fd18-bd63-4dee-a10f-cda566e0fd06 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:28.981190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1118/2211] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:28 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:28.985243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5d972-5d93-48f7-8097-12a4391bf316 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:28.996133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5d972-5d93-48f7-8097-12a4391bf316 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:29.061973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf65bb-29aa-4fb6-95ee-3255993f531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:29.062695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf65bb-29aa-4fb6-95ee-3255993f531e admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:29.063115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf65bb-29aa-4fb6-95ee-3255993f531e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:29.063396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf65bb-29aa-4fb6-95ee-3255993f531e admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:29.064291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf65bb-29aa-4fb6-95ee-3255993f531e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:29.066330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf65bb-29aa-4fb6-95ee-3255993f531e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:29.092400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1094/2212] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:28 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:41:29.096081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706fd18-bd63-4dee-a10f-cda566e0fd06 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:29.106550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706fd18-bd63-4dee-a10f-cda566e0fd06 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:29.111491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa5523-15fb-419c-99b1-30c375d0da1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:29.112154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa5523-15fb-419c-99b1-30c375d0da1d admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:29.112439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa5523-15fb-419c-99b1-30c375d0da1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:29.112721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa5523-15fb-419c-99b1-30c375d0da1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:29.113451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa5523-15fb-419c-99b1-30c375d0da1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:29.115161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa5523-15fb-419c-99b1-30c375d0da1d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:29.133578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1119/2213] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:29.138506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf65bb-29aa-4fb6-95ee-3255993f531e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:29.148497 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf65bb-29aa-4fb6-95ee-3255993f531e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:29.153135 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e555422-8b8a-4cf6-9cb3-a020ff9fb4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:29.154085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e555422-8b8a-4cf6-9cb3-a020ff9fb4d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:29.154369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e555422-8b8a-4cf6-9cb3-a020ff9fb4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:29.154647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e555422-8b8a-4cf6-9cb3-a020ff9fb4d6 admin admin] PATH_INFO: `/v3/projects/7759e80e6b6d4f96ac765525d4047962/users/d051ae21369f4f8f8119a2330cb011dd/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:29.170611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e555422-8b8a-4cf6-9cb3-a020ff9fb4d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7759e80e6b6d4f96ac765525d4047962, user_id=d051ae21369f4f8f8119a2330cb011dd, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:29.173297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e555422-8b8a-4cf6-9cb3-a020ff9fb4d6 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:29.186674 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1095/2214] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:29 2022] PUT /identity/v3/projects/7759e80e6b6d4f96ac765525d4047962/users/d051ae21369f4f8f8119a2330cb011dd/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:41:29.190644 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa5523-15fb-419c-99b1-30c375d0da1d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:29.201611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa5523-15fb-419c-99b1-30c375d0da1d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:29.258396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc220369-b134-4e55-b596-7e1d47b175a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:29.259124 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc220369-b134-4e55-b596-7e1d47b175a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:29.259434 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc220369-b134-4e55-b596-7e1d47b175a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:29.259715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc220369-b134-4e55-b596-7e1d47b175a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:29.260446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc220369-b134-4e55-b596-7e1d47b175a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:29.262080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc220369-b134-4e55-b596-7e1d47b175a8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:29.272829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1120/2215] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:29.277162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e555422-8b8a-4cf6-9cb3-a020ff9fb4d6 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:29.287683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e555422-8b8a-4cf6-9cb3-a020ff9fb4d6 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:29.292381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93f010-5b28-42d5-92a5-61e20eb4263b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:29.293266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93f010-5b28-42d5-92a5-61e20eb4263b admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:29.293546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93f010-5b28-42d5-92a5-61e20eb4263b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:29.293823 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93f010-5b28-42d5-92a5-61e20eb4263b admin admin] PATH_INFO: `/v3/projects/7759e80e6b6d4f96ac765525d4047962/users/d051ae21369f4f8f8119a2330cb011dd/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:29.295827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93f010-5b28-42d5-92a5-61e20eb4263b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7759e80e6b6d4f96ac765525d4047962, user_id=d051ae21369f4f8f8119a2330cb011dd, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:29.298516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93f010-5b28-42d5-92a5-61e20eb4263b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:29.310189 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1096/2216] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:29 2022] PUT /identity/v3/projects/7759e80e6b6d4f96ac765525d4047962/users/d051ae21369f4f8f8119a2330cb011dd/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:29.310520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc220369-b134-4e55-b596-7e1d47b175a8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:29.312308 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cc220369-b134-4e55-b596-7e1d47b175a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:29.321977 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12605e86-e068-42f2-b0b8-4b7cfebbf43a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:29.321977 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12605e86-e068-42f2-b0b8-4b7cfebbf43a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:29.321977 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12605e86-e068-42f2-b0b8-4b7cfebbf43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:29.325828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc220369-b134-4e55-b596-7e1d47b175a8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:29.348481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:29.358869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-12605e86-e068-42f2-b0b8-4b7cfebbf43a None None] MFA Rules not processed for user `d051ae21369f4f8f8119a2330cb011dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:29.360935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-12605e86-e068-42f2-b0b8-4b7cfebbf43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:29.390768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccac39d-e42f-4f5d-beb9-f8d3a8c65b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:29.391639 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccac39d-e42f-4f5d-beb9-f8d3a8c65b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:29.391920 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccac39d-e42f-4f5d-beb9-f8d3a8c65b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:29.392193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccac39d-e42f-4f5d-beb9-f8d3a8c65b88 admin admin] PATH_INFO: `/v3/users/308de2545b7d49698fefb004057a9dcb` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:29.393378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccac39d-e42f-4f5d-beb9-f8d3a8c65b88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=308de2545b7d49698fefb004057a9dcb)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:29.395352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccac39d-e42f-4f5d-beb9-f8d3a8c65b88 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:29.431546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1097/2217] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:41:29.432269 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:29.448495 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12605e86-e068-42f2-b0b8-4b7cfebbf43a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:29.449938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-12605e86-e068-42f2-b0b8-4b7cfebbf43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:29.451752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8ccac39d-e42f-4f5d-beb9-f8d3a8c65b88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '308de2545b7d49698fefb004057a9dcb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:29.461364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8ccac39d-e42f-4f5d-beb9-f8d3a8c65b88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '308de2545b7d49698fefb004057a9dcb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:29.462204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8ccac39d-e42f-4f5d-beb9-f8d3a8c65b88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '308de2545b7d49698fefb004057a9dcb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:29.470269 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8ccac39d-e42f-4f5d-beb9-f8d3a8c65b88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '308de2545b7d49698fefb004057a9dcb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:29.470954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8ccac39d-e42f-4f5d-beb9-f8d3a8c65b88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '308de2545b7d49698fefb004057a9dcb'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:29.491840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1121/2218] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:29 2022] DELETE /identity/v3/users/308de2545b7d49698fefb004057a9dcb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:29.496779 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1af14-7151-4821-98a4-1c82acfae3e1 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:29.499522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1af14-7151-4821-98a4-1c82acfae3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:29.499782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1af14-7151-4821-98a4-1c82acfae3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:29.511382 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12605e86-e068-42f2-b0b8-4b7cfebbf43a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:29.519369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-bfe1af14-7151-4821-98a4-1c82acfae3e1 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:29.520567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bfe1af14-7151-4821-98a4-1c82acfae3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:29.572631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6821e24-c470-495e-94ba-5958df152f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:29.573370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6821e24-c470-495e-94ba-5958df152f6e service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:29.573652 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6821e24-c470-495e-94ba-5958df152f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:29.574007 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6821e24-c470-495e-94ba-5958df152f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:29.576791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c6821e24-c470-495e-94ba-5958df152f6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:29.593292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1122/2219] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:41:29.602833 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1af14-7151-4821-98a4-1c82acfae3e1 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:29.604164 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bfe1af14-7151-4821-98a4-1c82acfae3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:29.657655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6821e24-c470-495e-94ba-5958df152f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:29.659150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6821e24-c470-495e-94ba-5958df152f6e service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:29.670121 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1098/2220] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:29 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:41:29.672602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1af14-7151-4821-98a4-1c82acfae3e1 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:29.676274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533fe29-92fb-4978-bb14-1856113e42b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:29.676921 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533fe29-92fb-4978-bb14-1856113e42b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:29.677203 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533fe29-92fb-4978-bb14-1856113e42b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:29.677506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533fe29-92fb-4978-bb14-1856113e42b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:29.686798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533fe29-92fb-4978-bb14-1856113e42b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:29.688225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533fe29-92fb-4978-bb14-1856113e42b3 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:29.697972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1123/2221] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:30.018408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6821e24-c470-495e-94ba-5958df152f6e service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:30.021023 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c6821e24-c470-495e-94ba-5958df152f6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:30.038044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:30.042843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6821e24-c470-495e-94ba-5958df152f6e service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:30.049062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:30.049905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.050238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.050563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] PATH_INFO: `/v3/projects/ddbe4a761eef4f7da9c514dff339f936` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.051685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddbe4a761eef4f7da9c514dff339f936)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:30.053668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:30.112105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddbe4a761eef4f7da9c514dff339f936', 'request_id': 'req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:30.118549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddbe4a761eef4f7da9c514dff339f936', 'request_id': 'req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:30.129018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddbe4a761eef4f7da9c514dff339f936', 'request_id': 'req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:30.143811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] The token cache is being invalidate because project ddbe4a761eef4f7da9c514dff339f936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:30.144101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:30.144742 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:30.146501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1099/2222] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:30 2022] DELETE /identity/v3/projects/ddbe4a761eef4f7da9c514dff339f936 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:30.151121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1824424-0dbb-41a1-bf56-e4ab4a6d688e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.151410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1824424-0dbb-41a1-bf56-e4ab4a6d688e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.152111 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1824424-0dbb-41a1-bf56-e4ab4a6d688e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.172941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:30.175154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:30.178836 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b1824424-0dbb-41a1-bf56-e4ab4a6d688e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:30.180088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b1824424-0dbb-41a1-bf56-e4ab4a6d688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:30.242414 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:30.259826 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c90b17-1d4a-47a3-95c7-1a1274a6f8cc admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:30.264710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:30.265600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.265880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.266184 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] PATH_INFO: `/v3/users/3c29e22e7f4a4e94a5200ea511dc5590` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.267409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c29e22e7f4a4e94a5200ea511dc5590)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:30.269545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:30.270029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1124/2223] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:30.271904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03233f-9e5b-4d7a-bca0-9aa7f75f8ca3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.272716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03233f-9e5b-4d7a-bca0-9aa7f75f8ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.273003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03233f-9e5b-4d7a-bca0-9aa7f75f8ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.298364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2e03233f-9e5b-4d7a-bca0-9aa7f75f8ca3 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:30.299587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2e03233f-9e5b-4d7a-bca0-9aa7f75f8ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:30.314669 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1125/2224] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:30.319666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03233f-9e5b-4d7a-bca0-9aa7f75f8ca3 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:30.328546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c29e22e7f4a4e94a5200ea511dc5590'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:30.329213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c29e22e7f4a4e94a5200ea511dc5590'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:30.329606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03233f-9e5b-4d7a-bca0-9aa7f75f8ca3 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:30.330188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c29e22e7f4a4e94a5200ea511dc5590'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:30.337363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510001ae-12f0-40e6-a6dc-891f0b0d959b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:30.337532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c29e22e7f4a4e94a5200ea511dc5590'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:30.338095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510001ae-12f0-40e6-a6dc-891f0b0d959b admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.338375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510001ae-12f0-40e6-a6dc-891f0b0d959b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.338653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510001ae-12f0-40e6-a6dc-891f0b0d959b admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.339549 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510001ae-12f0-40e6-a6dc-891f0b0d959b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:30.342226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510001ae-12f0-40e6-a6dc-891f0b0d959b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:30.342764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510001ae-12f0-40e6-a6dc-891f0b0d959b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:30.347913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c29e22e7f4a4e94a5200ea511dc5590'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:30.366732 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1100/2225] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:30 2022] DELETE /identity/v3/users/3c29e22e7f4a4e94a5200ea511dc5590 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:30.369900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1126/2226] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:30.371205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae07a-60cb-4d0e-b5bc-cfe2b04d82bd None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.371498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae07a-60cb-4d0e-b5bc-cfe2b04d82bd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.372410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae07a-60cb-4d0e-b5bc-cfe2b04d82bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.373646 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:30.375091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:30.382410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:30.396683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-08eae07a-60cb-4d0e-b5bc-cfe2b04d82bd None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:30.397851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-08eae07a-60cb-4d0e-b5bc-cfe2b04d82bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:30.449480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0c70d-f6a3-4170-9da1-a5b0ec61d794 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:30.457319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b59fe8-3c4d-489e-a37c-6c3a063abd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:30.458042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b59fe8-3c4d-489e-a37c-6c3a063abd29 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.458330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b59fe8-3c4d-489e-a37c-6c3a063abd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.458608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b59fe8-3c4d-489e-a37c-6c3a063abd29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.459309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b59fe8-3c4d-489e-a37c-6c3a063abd29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:30.461270 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b59fe8-3c4d-489e-a37c-6c3a063abd29 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:30.465207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1127/2227] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:41:30.475402 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae07a-60cb-4d0e-b5bc-cfe2b04d82bd None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:30.476915 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-08eae07a-60cb-4d0e-b5bc-cfe2b04d82bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:30.490707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1101/2228] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:30 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:41:30.494911 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b59fe8-3c4d-489e-a37c-6c3a063abd29 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:30.539182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae07a-60cb-4d0e-b5bc-cfe2b04d82bd None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:30.559980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b59fe8-3c4d-489e-a37c-6c3a063abd29 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:30.564860 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6cc1f-ec7c-40f3-8ff8-17506a882b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:30.565548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6cc1f-ec7c-40f3-8ff8-17506a882b73 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.565833 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6cc1f-ec7c-40f3-8ff8-17506a882b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.566142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6cc1f-ec7c-40f3-8ff8-17506a882b73 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.566847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6cc1f-ec7c-40f3-8ff8-17506a882b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:30.568785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6cc1f-ec7c-40f3-8ff8-17506a882b73 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:30.591094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1102/2229] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:30 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:30.600638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6cc1f-ec7c-40f3-8ff8-17506a882b73 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:30.608670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149947a-98df-4b9c-834c-8b7938413a4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:30.609577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149947a-98df-4b9c-834c-8b7938413a4a service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.609872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149947a-98df-4b9c-834c-8b7938413a4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.610184 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149947a-98df-4b9c-834c-8b7938413a4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.611723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6cc1f-ec7c-40f3-8ff8-17506a882b73 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:30.616942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146b517-3e86-4132-8685-5375bb0be5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:30.617849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146b517-3e86-4132-8685-5375bb0be5ac admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.618005 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146b517-3e86-4132-8685-5375bb0be5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.618655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146b517-3e86-4132-8685-5375bb0be5ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.618936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146b517-3e86-4132-8685-5375bb0be5ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:30.620514 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146b517-3e86-4132-8685-5375bb0be5ac admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:30.620904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149947a-98df-4b9c-834c-8b7938413a4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:30.622864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149947a-98df-4b9c-834c-8b7938413a4a service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:30.636552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1103/2230] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:30.636645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1128/2231] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:30.642582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146b517-3e86-4132-8685-5375bb0be5ac admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:30.652788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146b517-3e86-4132-8685-5375bb0be5ac admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:30.658039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36241b79-9378-4aca-b440-7858b064e0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:30.658702 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241b79-9378-4aca-b440-7858b064e0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.658983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241b79-9378-4aca-b440-7858b064e0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.659261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241b79-9378-4aca-b440-7858b064e0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.660124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36241b79-9378-4aca-b440-7858b064e0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:30.661749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36241b79-9378-4aca-b440-7858b064e0a0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:30.675753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1104/2232] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:30.679402 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149947a-98df-4b9c-834c-8b7938413a4a service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:30.690119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149947a-98df-4b9c-834c-8b7938413a4a service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:30.694834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527886b2-52ca-48bc-8896-96d4c4c41e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:30.695806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527886b2-52ca-48bc-8896-96d4c4c41e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.696079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527886b2-52ca-48bc-8896-96d4c4c41e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.696403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527886b2-52ca-48bc-8896-96d4c4c41e22 admin admin] PATH_INFO: `/v3/projects/d1530a1e5bf24475bb0bf9725401c09a/users/ab5e3c513d1b458d9620e575265ddb83/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.712102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527886b2-52ca-48bc-8896-96d4c4c41e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1530a1e5bf24475bb0bf9725401c09a, user_id=ab5e3c513d1b458d9620e575265ddb83, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:30.714844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527886b2-52ca-48bc-8896-96d4c4c41e22 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:30.728885 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1129/2233] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:30 2022] PUT /identity/v3/projects/d1530a1e5bf24475bb0bf9725401c09a/users/ab5e3c513d1b458d9620e575265ddb83/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:30.802409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff123b3a-3b0f-4851-aa52-8894ba3b4237 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:30.802702 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff123b3a-3b0f-4851-aa52-8894ba3b4237 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:30.802977 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff123b3a-3b0f-4851-aa52-8894ba3b4237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:30.832536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-ff123b3a-3b0f-4851-aa52-8894ba3b4237 None None] MFA Rules not processed for user `ab5e3c513d1b458d9620e575265ddb83`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:30.833744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ff123b3a-3b0f-4851-aa52-8894ba3b4237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:30.888507 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:30.911869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1105/2234] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:30.924753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527886b2-52ca-48bc-8896-96d4c4c41e22 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:30.926835 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-527886b2-52ca-48bc-8896-96d4c4c41e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:30.940868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:30.941895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff123b3a-3b0f-4851-aa52-8894ba3b4237 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:30.944851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ff123b3a-3b0f-4851-aa52-8894ba3b4237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:31.026100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527886b2-52ca-48bc-8896-96d4c4c41e22 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:31.026657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff123b3a-3b0f-4851-aa52-8894ba3b4237 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:31.030869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699fdf2-9bc5-4831-a556-d8228e67a35a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:31.031526 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699fdf2-9bc5-4831-a556-d8228e67a35a service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:31.031807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699fdf2-9bc5-4831-a556-d8228e67a35a service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:31.032086 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699fdf2-9bc5-4831-a556-d8228e67a35a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:31.032281 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:31.033057 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:31.033342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:31.033619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] PATH_INFO: `/v3/projects/10c02ecd38ca45279ab37bf17cc78c82` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:31.034850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10c02ecd38ca45279ab37bf17cc78c82)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:31.036777 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:31.040884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699fdf2-9bc5-4831-a556-d8228e67a35a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:31.042359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699fdf2-9bc5-4831-a556-d8228e67a35a service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:31.051304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1130/2235] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:31.090557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10c02ecd38ca45279ab37bf17cc78c82', 'request_id': 'req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.097627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10c02ecd38ca45279ab37bf17cc78c82', 'request_id': 'req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.104565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10c02ecd38ca45279ab37bf17cc78c82', 'request_id': 'req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.119070 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] The token cache is being invalidate because project 10c02ecd38ca45279ab37bf17cc78c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:31.119359 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.120073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.121809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1106/2236] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:30 2022] DELETE /identity/v3/projects/10c02ecd38ca45279ab37bf17cc78c82 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:41:31.125824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699fdf2-9bc5-4831-a556-d8228e67a35a service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:31.136857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699fdf2-9bc5-4831-a556-d8228e67a35a service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:31.202079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e1f51-86d7-465a-b841-1400aed7454c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:31.202964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e1f51-86d7-465a-b841-1400aed7454c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:31.203246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e1f51-86d7-465a-b841-1400aed7454c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:31.203518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e1f51-86d7-465a-b841-1400aed7454c admin admin] PATH_INFO: `/v3/users/d051ae21369f4f8f8119a2330cb011dd` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:31.204796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358e1f51-86d7-465a-b841-1400aed7454c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d051ae21369f4f8f8119a2330cb011dd)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:31.206790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358e1f51-86d7-465a-b841-1400aed7454c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:31.257643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-358e1f51-86d7-465a-b841-1400aed7454c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd051ae21369f4f8f8119a2330cb011dd'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.266952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-358e1f51-86d7-465a-b841-1400aed7454c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd051ae21369f4f8f8119a2330cb011dd'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.267709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-358e1f51-86d7-465a-b841-1400aed7454c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd051ae21369f4f8f8119a2330cb011dd'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.274943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-358e1f51-86d7-465a-b841-1400aed7454c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd051ae21369f4f8f8119a2330cb011dd'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.275612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-358e1f51-86d7-465a-b841-1400aed7454c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd051ae21369f4f8f8119a2330cb011dd'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.299390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1131/2237] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:31 2022] DELETE /identity/v3/users/d051ae21369f4f8f8119a2330cb011dd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:31.626336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:31.628166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:31.642574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbe83f-95fb-44b8-a378-67a1bf1c9124 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:31.703018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:31.703957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:31.704248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:31.704525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418 admin admin] PATH_INFO: `/v3/projects/7759e80e6b6d4f96ac765525d4047962` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:31.705735 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7759e80e6b6d4f96ac765525d4047962)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:31.707675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:31.728445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:31.754975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7759e80e6b6d4f96ac765525d4047962', 'request_id': 'req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.761007 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7759e80e6b6d4f96ac765525d4047962', 'request_id': 'req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.767534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7759e80e6b6d4f96ac765525d4047962', 'request_id': 'req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.780504 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418 admin admin] The token cache is being invalidate because project 7759e80e6b6d4f96ac765525d4047962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:31.780789 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.781406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-5ece5c77-b634-4fc6-8cd3-5e2f1dc90418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:31.782951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1107/2238] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:31 2022] DELETE /identity/v3/projects/7759e80e6b6d4f96ac765525d4047962 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:31.787883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978ad50-67c7-4c1f-bcd5-4204359d6057 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:31.788364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978ad50-67c7-4c1f-bcd5-4204359d6057 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:31.788477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978ad50-67c7-4c1f-bcd5-4204359d6057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:31.807550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6978ad50-67c7-4c1f-bcd5-4204359d6057 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:31.808747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6978ad50-67c7-4c1f-bcd5-4204359d6057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:31.834528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:31.873630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1132/2239] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:31.881315 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15f1bb-9e50-4b90-8a93-497f51b2975e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:31.881599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15f1bb-9e50-4b90-8a93-497f51b2975e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:31.881875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15f1bb-9e50-4b90-8a93-497f51b2975e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:31.901484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-5a15f1bb-9e50-4b90-8a93-497f51b2975e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:31.902701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5a15f1bb-9e50-4b90-8a93-497f51b2975e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:31.911271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1108/2240] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:31.915490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978ad50-67c7-4c1f-bcd5-4204359d6057 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:31.925376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978ad50-67c7-4c1f-bcd5-4204359d6057 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:31.930173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef9aae-dac3-41e7-89f0-e2f276e19982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:31.930827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef9aae-dac3-41e7-89f0-e2f276e19982 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:31.931112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef9aae-dac3-41e7-89f0-e2f276e19982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:31.931390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef9aae-dac3-41e7-89f0-e2f276e19982 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:31.932122 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef9aae-dac3-41e7-89f0-e2f276e19982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:31.932434 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef9aae-dac3-41e7-89f0-e2f276e19982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:31.934077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef9aae-dac3-41e7-89f0-e2f276e19982 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:31.946264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1133/2241] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:31.951092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15f1bb-9e50-4b90-8a93-497f51b2975e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:31.961010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15f1bb-9e50-4b90-8a93-497f51b2975e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:31.966686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbbd9b-65dc-46d9-acf0-7ed3c3377982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:31.967356 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbbd9b-65dc-46d9-acf0-7ed3c3377982 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:31.967637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbbd9b-65dc-46d9-acf0-7ed3c3377982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:31.967908 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbbd9b-65dc-46d9-acf0-7ed3c3377982 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:31.968617 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbbd9b-65dc-46d9-acf0-7ed3c3377982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:31.970552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbbd9b-65dc-46d9-acf0-7ed3c3377982 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:32.003203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1109/2242] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:31 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:32.007667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef9aae-dac3-41e7-89f0-e2f276e19982 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:32.018230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef9aae-dac3-41e7-89f0-e2f276e19982 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:32.075925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77201354-def8-4c7c-bb2b-d486a8c48f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:32.076604 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77201354-def8-4c7c-bb2b-d486a8c48f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.076889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77201354-def8-4c7c-bb2b-d486a8c48f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.077167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77201354-def8-4c7c-bb2b-d486a8c48f69 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.077860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77201354-def8-4c7c-bb2b-d486a8c48f69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:32.079822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77201354-def8-4c7c-bb2b-d486a8c48f69 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:32.101924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1134/2243] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:32 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:41:32.106915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbbd9b-65dc-46d9-acf0-7ed3c3377982 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:32.117817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbbd9b-65dc-46d9-acf0-7ed3c3377982 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:32.123125 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b9be0-d840-477e-a5cc-a8f75625f70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:32.123847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b9be0-d840-477e-a5cc-a8f75625f70e admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.124078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b9be0-d840-477e-a5cc-a8f75625f70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.124372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b9be0-d840-477e-a5cc-a8f75625f70e admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.125120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b9be0-d840-477e-a5cc-a8f75625f70e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:32.126764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b9be0-d840-477e-a5cc-a8f75625f70e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:32.137288 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1110/2244] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:32.141744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77201354-def8-4c7c-bb2b-d486a8c48f69 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:32.151614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77201354-def8-4c7c-bb2b-d486a8c48f69 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:32.156989 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c900abc-caeb-42b7-add3-93dc97f27af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:32.157664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c900abc-caeb-42b7-add3-93dc97f27af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.157984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c900abc-caeb-42b7-add3-93dc97f27af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.158287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c900abc-caeb-42b7-add3-93dc97f27af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.159017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c900abc-caeb-42b7-add3-93dc97f27af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:32.160625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c900abc-caeb-42b7-add3-93dc97f27af0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:32.171943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1135/2245] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:32.176174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b9be0-d840-477e-a5cc-a8f75625f70e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:32.186772 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b9be0-d840-477e-a5cc-a8f75625f70e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:32.198364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb96f1c-d4fb-47c0-9bd8-ff903007cdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:32.199276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb96f1c-d4fb-47c0-9bd8-ff903007cdd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.199554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb96f1c-d4fb-47c0-9bd8-ff903007cdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.199822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb96f1c-d4fb-47c0-9bd8-ff903007cdd8 admin admin] PATH_INFO: `/v3/projects/9cfcd2f14bda43e9986f88e054c6a228/users/0bb78c107ed34d9d92b03498dbab2589/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.216316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb96f1c-d4fb-47c0-9bd8-ff903007cdd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cfcd2f14bda43e9986f88e054c6a228, user_id=0bb78c107ed34d9d92b03498dbab2589, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:32.219007 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb96f1c-d4fb-47c0-9bd8-ff903007cdd8 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:32.231478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1111/2246] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:32 2022] PUT /identity/v3/projects/9cfcd2f14bda43e9986f88e054c6a228/users/0bb78c107ed34d9d92b03498dbab2589/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:41:32.242946 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5216b58-f04a-4181-ace1-1b5d6ef3997c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.243235 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5216b58-f04a-4181-ace1-1b5d6ef3997c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.243515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5216b58-f04a-4181-ace1-1b5d6ef3997c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.275238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b5216b58-f04a-4181-ace1-1b5d6ef3997c None None] MFA Rules not processed for user `0bb78c107ed34d9d92b03498dbab2589`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:32.276462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b5216b58-f04a-4181-ace1-1b5d6ef3997c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:32.342536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1136/2247] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:32.347064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb96f1c-d4fb-47c0-9bd8-ff903007cdd8 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:32.411647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb96f1c-d4fb-47c0-9bd8-ff903007cdd8 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:32.416362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14da520-642d-49ac-8d30-a53c037a1bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:32.417015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14da520-642d-49ac-8d30-a53c037a1bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.417297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14da520-642d-49ac-8d30-a53c037a1bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.417577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14da520-642d-49ac-8d30-a53c037a1bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.418315 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14da520-642d-49ac-8d30-a53c037a1bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:32.420720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14da520-642d-49ac-8d30-a53c037a1bbd admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:32.447715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1112/2248] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:32 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:41:32.453302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5216b58-f04a-4181-ace1-1b5d6ef3997c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:32.464224 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5216b58-f04a-4181-ace1-1b5d6ef3997c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:32.484087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:32.529306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6172-f9f9-4a42-b148-8d73b2c98cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:32.530003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6172-f9f9-4a42-b148-8d73b2c98cbb admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.530287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6172-f9f9-4a42-b148-8d73b2c98cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.530572 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6172-f9f9-4a42-b148-8d73b2c98cbb admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.531285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d6172-f9f9-4a42-b148-8d73b2c98cbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:32.533211 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d6172-f9f9-4a42-b148-8d73b2c98cbb admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:32.555701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1137/2249] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:32 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:32.560561 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14da520-642d-49ac-8d30-a53c037a1bbd admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:32.571352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14da520-642d-49ac-8d30-a53c037a1bbd admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:32.575986 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fed837-5e35-4817-8c46-577894629266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:32.576662 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fed837-5e35-4817-8c46-577894629266 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.576943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fed837-5e35-4817-8c46-577894629266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.577227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fed837-5e35-4817-8c46-577894629266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.577983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fed837-5e35-4817-8c46-577894629266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:32.579584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fed837-5e35-4817-8c46-577894629266 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:32.590626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1113/2250] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:41:32.594826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6172-f9f9-4a42-b148-8d73b2c98cbb admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:32.604589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6172-f9f9-4a42-b148-8d73b2c98cbb admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:32.610465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6e010-0e7f-4873-ab1d-6979f73b1369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:32.613016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6e010-0e7f-4873-ab1d-6979f73b1369 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.613307 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6e010-0e7f-4873-ab1d-6979f73b1369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.613589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6e010-0e7f-4873-ab1d-6979f73b1369 admin admin] PATH_INFO: `/v3/projects/6a261d56105247879aaf114228913a8f/users/71c9ab5059e948c4b46b91184b3e8f3c/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.629618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6e010-0e7f-4873-ab1d-6979f73b1369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a261d56105247879aaf114228913a8f, user_id=71c9ab5059e948c4b46b91184b3e8f3c, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:32.632401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6e010-0e7f-4873-ab1d-6979f73b1369 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:32.645233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1138/2251] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:32 2022] PUT /identity/v3/projects/6a261d56105247879aaf114228913a8f/users/71c9ab5059e948c4b46b91184b3e8f3c/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:32.648993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fed837-5e35-4817-8c46-577894629266 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:32.659092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fed837-5e35-4817-8c46-577894629266 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:32.672886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:32.728473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039fda6-2f84-419e-9a6f-1b32afaf686b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:32.729227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039fda6-2f84-419e-9a6f-1b32afaf686b admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.729574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039fda6-2f84-419e-9a6f-1b32afaf686b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.729939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039fda6-2f84-419e-9a6f-1b32afaf686b admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.730746 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039fda6-2f84-419e-9a6f-1b32afaf686b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:32.733017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039fda6-2f84-419e-9a6f-1b32afaf686b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:32.747032 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1114/2252] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:32.751125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6e010-0e7f-4873-ab1d-6979f73b1369 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:32.762828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6e010-0e7f-4873-ab1d-6979f73b1369 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:32.773957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433183bf-3849-449a-ba66-9cfa18955610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:32.774844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433183bf-3849-449a-ba66-9cfa18955610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.775136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433183bf-3849-449a-ba66-9cfa18955610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.775439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433183bf-3849-449a-ba66-9cfa18955610 admin admin] PATH_INFO: `/v3/projects/6a261d56105247879aaf114228913a8f/users/71c9ab5059e948c4b46b91184b3e8f3c/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.777622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433183bf-3849-449a-ba66-9cfa18955610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a261d56105247879aaf114228913a8f, user_id=71c9ab5059e948c4b46b91184b3e8f3c, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:32.781947 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433183bf-3849-449a-ba66-9cfa18955610 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:32.799110 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1139/2253] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:32 2022] PUT /identity/v3/projects/6a261d56105247879aaf114228913a8f/users/71c9ab5059e948c4b46b91184b3e8f3c/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:32.810519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6f8f3-bd71-4538-973b-69a65179fc76 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.810812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6f8f3-bd71-4538-973b-69a65179fc76 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.811092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6f8f3-bd71-4538-973b-69a65179fc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.839262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-26e6f8f3-bd71-4538-973b-69a65179fc76 None None] MFA Rules not processed for user `71c9ab5059e948c4b46b91184b3e8f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:32.840996 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-26e6f8f3-bd71-4538-973b-69a65179fc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:32.909395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1115/2254] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:32.922012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433183bf-3849-449a-ba66-9cfa18955610 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:32.923712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-433183bf-3849-449a-ba66-9cfa18955610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:32.983345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433183bf-3849-449a-ba66-9cfa18955610 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:32.987245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd877d30-68b7-4e38-ba77-1c14a0a0c828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:32.987890 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd877d30-68b7-4e38-ba77-1c14a0a0c828 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:32.988173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd877d30-68b7-4e38-ba77-1c14a0a0c828 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:32.988447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd877d30-68b7-4e38-ba77-1c14a0a0c828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:32.997279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd877d30-68b7-4e38-ba77-1c14a0a0c828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:32.998745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd877d30-68b7-4e38-ba77-1c14a0a0c828 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:33.008026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1140/2255] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:33.328452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6f8f3-bd71-4538-973b-69a65179fc76 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:33.338129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6f8f3-bd71-4538-973b-69a65179fc76 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:33.342884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977d07b-839e-411e-96d9-a1a4f2664788 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c9ab5059e948c4b46b91184b3e8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a261d56105247879aaf114228913a8f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:33.343552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977d07b-839e-411e-96d9-a1a4f2664788 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:33.343835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977d07b-839e-411e-96d9-a1a4f2664788 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:33.344111 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977d07b-839e-411e-96d9-a1a4f2664788 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:33.344851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977d07b-839e-411e-96d9-a1a4f2664788 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:33.345154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977d07b-839e-411e-96d9-a1a4f2664788 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:33.346796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977d07b-839e-411e-96d9-a1a4f2664788 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:33.360067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1116/2256] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:33.364362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd877d30-68b7-4e38-ba77-1c14a0a0c828 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:33.375088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd877d30-68b7-4e38-ba77-1c14a0a0c828 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:33.380098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1876604-c0f8-410f-9d9b-28c2ad53f988 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c9ab5059e948c4b46b91184b3e8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a261d56105247879aaf114228913a8f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:33.380756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1876604-c0f8-410f-9d9b-28c2ad53f988 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:33.381044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1876604-c0f8-410f-9d9b-28c2ad53f988 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:33.381324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1876604-c0f8-410f-9d9b-28c2ad53f988 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:33.382070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1876604-c0f8-410f-9d9b-28c2ad53f988 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:33.383984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1876604-c0f8-410f-9d9b-28c2ad53f988 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:33.410634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1141/2257] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:33 2022] POST /identity/v3/projects => generated 353 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:33.414862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977d07b-839e-411e-96d9-a1a4f2664788 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:33.425166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977d07b-839e-411e-96d9-a1a4f2664788 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:33.484752 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121dcb0-caaa-428e-9700-485370f93b3f tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c9ab5059e948c4b46b91184b3e8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a261d56105247879aaf114228913a8f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:33.485348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121dcb0-caaa-428e-9700-485370f93b3f tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:33.485642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121dcb0-caaa-428e-9700-485370f93b3f tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:33.485961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121dcb0-caaa-428e-9700-485370f93b3f tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:33.486722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121dcb0-caaa-428e-9700-485370f93b3f tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:33.487041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121dcb0-caaa-428e-9700-485370f93b3f tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:33.488697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121dcb0-caaa-428e-9700-485370f93b3f tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:33.502244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1117/2258] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:33.540589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1876604-c0f8-410f-9d9b-28c2ad53f988 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:33.552397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1876604-c0f8-410f-9d9b-28c2ad53f988 tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:33.557071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c9ab5059e948c4b46b91184b3e8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a261d56105247879aaf114228913a8f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:33.558047 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:33.558420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:33.558702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] PATH_INFO: `/v3/projects/6058b8938137490fb620ace319affe18` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:33.560134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6058b8938137490fb620ace319affe18)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:33.562199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:33.569303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121dcb0-caaa-428e-9700-485370f93b3f tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:33.571066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0121dcb0-caaa-428e-9700-485370f93b3f tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:33.613338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:33.620199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6058b8938137490fb620ace319affe18', 'request_id': 'req-c20af35e-291d-4847-83f6-d177f6f3101b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:33.623393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:INSERT=7,keystone:DELETE=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:33.632106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6058b8938137490fb620ace319affe18', 'request_id': 'req-c20af35e-291d-4847-83f6-d177f6f3101b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:33.640237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6058b8938137490fb620ace319affe18', 'request_id': 'req-c20af35e-291d-4847-83f6-d177f6f3101b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:33.652122 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] The token cache is being invalidate because project 6058b8938137490fb620ace319affe18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:33.652485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:33.653857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:33.655786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1142/2259] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:33 2022] DELETE /identity/v3/projects/6058b8938137490fb620ace319affe18 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:33.673952 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121dcb0-caaa-428e-9700-485370f93b3f tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:33.736088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9f430-a91f-453b-a9da-e4daf5d308c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b07dcd8c0de4d1694d4363520726df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:33.736672 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9f430-a91f-453b-a9da-e4daf5d308c7 service glance] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:33.736955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9f430-a91f-453b-a9da-e4daf5d308c7 service glance] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:33.737230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9f430-a91f-453b-a9da-e4daf5d308c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:33.739548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-28c9f430-a91f-453b-a9da-e4daf5d308c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:33.812889 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9f430-a91f-453b-a9da-e4daf5d308c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:33.814966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9f430-a91f-453b-a9da-e4daf5d308c7 service glance] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:33.832678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1118/2260] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:33 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:41:34.013831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:34.015483 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:34.108615 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20af35e-291d-4847-83f6-d177f6f3101b tempest-VolumeQuotaClassesTest-328886208 tempest-VolumeQuotaClassesTest-328886208-project-admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:34.113975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:34.114944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:34.115322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:34.115696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] PATH_INFO: `/v3/users/0bb78c107ed34d9d92b03498dbab2589` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:34.116990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb78c107ed34d9d92b03498dbab2589)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:34.119081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:34.174702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb78c107ed34d9d92b03498dbab2589'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.182846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb78c107ed34d9d92b03498dbab2589'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.183692 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb78c107ed34d9d92b03498dbab2589'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.191501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb78c107ed34d9d92b03498dbab2589'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.192202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb78c107ed34d9d92b03498dbab2589'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.210509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1143/2261] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:34 2022] DELETE /identity/v3/users/0bb78c107ed34d9d92b03498dbab2589 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:34.215229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea0a15-5323-4939-aa64-c09a9f91594a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:34.216096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea0a15-5323-4939-aa64-c09a9f91594a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:34.216378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea0a15-5323-4939-aa64-c09a9f91594a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:34.236500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b6ea0a15-5323-4939-aa64-c09a9f91594a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:34.237629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b6ea0a15-5323-4939-aa64-c09a9f91594a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:34.299372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1119/2262] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:34.311181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:34.312614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:34.378559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bffdf-93b7-47b3-98c3-d2b739b58433 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:34.382436 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e7c89-79e8-46f6-87d8-5d1adb37e43c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:34.383089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e7c89-79e8-46f6-87d8-5d1adb37e43c service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:34.383377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e7c89-79e8-46f6-87d8-5d1adb37e43c service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:34.383655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e7c89-79e8-46f6-87d8-5d1adb37e43c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:34.394338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e7c89-79e8-46f6-87d8-5d1adb37e43c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:34.395781 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e7c89-79e8-46f6-87d8-5d1adb37e43c service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:34.406618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1144/2263] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:34.694505 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea0a15-5323-4939-aa64-c09a9f91594a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:34.695947 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b6ea0a15-5323-4939-aa64-c09a9f91594a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:34.709809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea0a15-5323-4939-aa64-c09a9f91594a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:34.714811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:34.716311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:34.716604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:34.717262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] PATH_INFO: `/v3/projects/9cfcd2f14bda43e9986f88e054c6a228` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:34.718489 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cfcd2f14bda43e9986f88e054c6a228)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:34.720566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:34.753400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:34.771007 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cfcd2f14bda43e9986f88e054c6a228', 'request_id': 'req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.777132 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cfcd2f14bda43e9986f88e054c6a228', 'request_id': 'req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.782737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cfcd2f14bda43e9986f88e054c6a228', 'request_id': 'req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.795784 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] The token cache is being invalidate because project 9cfcd2f14bda43e9986f88e054c6a228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:34.796064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.796678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.798372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1120/2264] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:34 2022] DELETE /identity/v3/projects/9cfcd2f14bda43e9986f88e054c6a228 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:34.802401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e7c89-79e8-46f6-87d8-5d1adb37e43c service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:34.807461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:34.814359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e7c89-79e8-46f6-87d8-5d1adb37e43c service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:34.875987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9759490-bdce-4ff9-9e49-405617f3b10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:34.876886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9759490-bdce-4ff9-9e49-405617f3b10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:34.877160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9759490-bdce-4ff9-9e49-405617f3b10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:34.877428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9759490-bdce-4ff9-9e49-405617f3b10a admin admin] PATH_INFO: `/v3/users/71c9ab5059e948c4b46b91184b3e8f3c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:34.878618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9759490-bdce-4ff9-9e49-405617f3b10a admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c9ab5059e948c4b46b91184b3e8f3c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:34.880528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9759490-bdce-4ff9-9e49-405617f3b10a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:34.929231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d9759490-bdce-4ff9-9e49-405617f3b10a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c9ab5059e948c4b46b91184b3e8f3c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.939728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d9759490-bdce-4ff9-9e49-405617f3b10a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c9ab5059e948c4b46b91184b3e8f3c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.940378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d9759490-bdce-4ff9-9e49-405617f3b10a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c9ab5059e948c4b46b91184b3e8f3c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.948286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d9759490-bdce-4ff9-9e49-405617f3b10a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c9ab5059e948c4b46b91184b3e8f3c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.948944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d9759490-bdce-4ff9-9e49-405617f3b10a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c9ab5059e948c4b46b91184b3e8f3c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:34.970084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1145/2265] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:34 2022] DELETE /identity/v3/users/71c9ab5059e948c4b46b91184b3e8f3c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:35.336017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:35.337440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:35.351463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ffb22-b03a-4d89-809e-2c8e4cfbe6ab admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:35.417941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d642ba-da98-4367-9df9-855e23cab9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:35.419248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d642ba-da98-4367-9df9-855e23cab9ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:35.419590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d642ba-da98-4367-9df9-855e23cab9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:35.419991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d642ba-da98-4367-9df9-855e23cab9ee admin admin] PATH_INFO: `/v3/projects/6a261d56105247879aaf114228913a8f` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:35.421596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d642ba-da98-4367-9df9-855e23cab9ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a261d56105247879aaf114228913a8f)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:35.424391 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d642ba-da98-4367-9df9-855e23cab9ee admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:35.477605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-46d642ba-da98-4367-9df9-855e23cab9ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a261d56105247879aaf114228913a8f', 'request_id': 'req-46d642ba-da98-4367-9df9-855e23cab9ee'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:35.483772 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-46d642ba-da98-4367-9df9-855e23cab9ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a261d56105247879aaf114228913a8f', 'request_id': 'req-46d642ba-da98-4367-9df9-855e23cab9ee'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:35.489682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-46d642ba-da98-4367-9df9-855e23cab9ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a261d56105247879aaf114228913a8f', 'request_id': 'req-46d642ba-da98-4367-9df9-855e23cab9ee'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:35.503120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-46d642ba-da98-4367-9df9-855e23cab9ee admin admin] The token cache is being invalidate because project 6a261d56105247879aaf114228913a8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:35.503406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-46d642ba-da98-4367-9df9-855e23cab9ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:35.504025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-46d642ba-da98-4367-9df9-855e23cab9ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:35.505718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1121/2266] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:35 2022] DELETE /identity/v3/projects/6a261d56105247879aaf114228913a8f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:35.510763 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80e696-aaad-4e9e-a7ca-1ef91f3bd285 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:35.511276 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80e696-aaad-4e9e-a7ca-1ef91f3bd285 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:35.511837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80e696-aaad-4e9e-a7ca-1ef91f3bd285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:35.536552 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3d80e696-aaad-4e9e-a7ca-1ef91f3bd285 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:35.537779 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3d80e696-aaad-4e9e-a7ca-1ef91f3bd285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:35.600447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1146/2267] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:35.609705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10ce94-d7b5-435d-b454-cfa9f4daaf05 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:35.610041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10ce94-d7b5-435d-b454-cfa9f4daaf05 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:35.610297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10ce94-d7b5-435d-b454-cfa9f4daaf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:35.629830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-1b10ce94-d7b5-435d-b454-cfa9f4daaf05 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:35.631067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-1b10ce94-d7b5-435d-b454-cfa9f4daaf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:35.648167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1122/2268] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:41:35.652569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80e696-aaad-4e9e-a7ca-1ef91f3bd285 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:35.663994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80e696-aaad-4e9e-a7ca-1ef91f3bd285 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:35.668677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70d0ce-2409-4726-9bff-f3b1d4e71f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:35.669350 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70d0ce-2409-4726-9bff-f3b1d4e71f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:35.669634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70d0ce-2409-4726-9bff-f3b1d4e71f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:35.669927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70d0ce-2409-4726-9bff-f3b1d4e71f39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:35.670677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70d0ce-2409-4726-9bff-f3b1d4e71f39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:35.670989 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70d0ce-2409-4726-9bff-f3b1d4e71f39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:35.672594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70d0ce-2409-4726-9bff-f3b1d4e71f39 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:35.684397 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1147/2269] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:35.688784 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10ce94-d7b5-435d-b454-cfa9f4daaf05 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:35.698055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10ce94-d7b5-435d-b454-cfa9f4daaf05 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:35.703298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdc3f3-d931-43da-93db-fc12b9ca2405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:35.703943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdc3f3-d931-43da-93db-fc12b9ca2405 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:35.704226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdc3f3-d931-43da-93db-fc12b9ca2405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:35.704503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdc3f3-d931-43da-93db-fc12b9ca2405 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:35.705244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdc3f3-d931-43da-93db-fc12b9ca2405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:35.707273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdc3f3-d931-43da-93db-fc12b9ca2405 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:35.735641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1123/2270] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:35 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:35.739883 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70d0ce-2409-4726-9bff-f3b1d4e71f39 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:35.750325 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70d0ce-2409-4726-9bff-f3b1d4e71f39 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:35.754864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:35.810622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e640fc2-f6d4-4b6d-bd2e-5bb5154397a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:35.811295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e640fc2-f6d4-4b6d-bd2e-5bb5154397a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:35.811580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e640fc2-f6d4-4b6d-bd2e-5bb5154397a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:35.811859 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e640fc2-f6d4-4b6d-bd2e-5bb5154397a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:35.812557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e640fc2-f6d4-4b6d-bd2e-5bb5154397a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:35.814535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e640fc2-f6d4-4b6d-bd2e-5bb5154397a0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:35.838230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1148/2271] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:35 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:35.839418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdc3f3-d931-43da-93db-fc12b9ca2405 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:35.849563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdc3f3-d931-43da-93db-fc12b9ca2405 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:35.854284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff223c-ed4d-4b0d-8951-df3024a94d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:35.854889 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff223c-ed4d-4b0d-8951-df3024a94d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:35.855176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff223c-ed4d-4b0d-8951-df3024a94d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:35.855453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff223c-ed4d-4b0d-8951-df3024a94d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:35.856220 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff223c-ed4d-4b0d-8951-df3024a94d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:35.857846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff223c-ed4d-4b0d-8951-df3024a94d68 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:35.867954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1124/2272] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:35.872201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e640fc2-f6d4-4b6d-bd2e-5bb5154397a0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:35.882081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e640fc2-f6d4-4b6d-bd2e-5bb5154397a0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:35.886682 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b284b4-c735-4c2e-8389-1e21cb95fede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:35.887962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b284b4-c735-4c2e-8389-1e21cb95fede admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:35.888252 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b284b4-c735-4c2e-8389-1e21cb95fede admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:35.888531 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b284b4-c735-4c2e-8389-1e21cb95fede admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:35.889251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b284b4-c735-4c2e-8389-1e21cb95fede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:35.890885 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b284b4-c735-4c2e-8389-1e21cb95fede admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:35.901222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1149/2273] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:35.905136 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff223c-ed4d-4b0d-8951-df3024a94d68 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:35.911427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:35.915124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff223c-ed4d-4b0d-8951-df3024a94d68 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:35.921253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb12bc1-c427-464d-9c01-49c127e6f218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:35.922342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb12bc1-c427-464d-9c01-49c127e6f218 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:35.922786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb12bc1-c427-464d-9c01-49c127e6f218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:35.923124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb12bc1-c427-464d-9c01-49c127e6f218 admin admin] PATH_INFO: `/v3/projects/df5a90f4ca324da083b7bd2fe5941f91/users/d3d5e2c2e13a4d6297e0dc263e1404a3/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:35.938236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb12bc1-c427-464d-9c01-49c127e6f218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5a90f4ca324da083b7bd2fe5941f91, user_id=d3d5e2c2e13a4d6297e0dc263e1404a3, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:35.940888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb12bc1-c427-464d-9c01-49c127e6f218 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:35.956484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1125/2274] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:35 2022] PUT /identity/v3/projects/df5a90f4ca324da083b7bd2fe5941f91/users/d3d5e2c2e13a4d6297e0dc263e1404a3/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:35.967843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e447238-9cc0-49b4-b493-99f7640cb359 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:35.968133 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e447238-9cc0-49b4-b493-99f7640cb359 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:35.968410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e447238-9cc0-49b4-b493-99f7640cb359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:35.997162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5e447238-9cc0-49b4-b493-99f7640cb359 None None] MFA Rules not processed for user `d3d5e2c2e13a4d6297e0dc263e1404a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:35.998395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5e447238-9cc0-49b4-b493-99f7640cb359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:36.061672 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1150/2275] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:36.066356 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb12bc1-c427-464d-9c01-49c127e6f218 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:36.130437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb12bc1-c427-464d-9c01-49c127e6f218 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:36.134986 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f847ad3-3789-442b-8971-e2771797bf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:36.135643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f847ad3-3789-442b-8971-e2771797bf4f admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:36.135925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f847ad3-3789-442b-8971-e2771797bf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:36.136200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f847ad3-3789-442b-8971-e2771797bf4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:36.136904 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f847ad3-3789-442b-8971-e2771797bf4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:36.138944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f847ad3-3789-442b-8971-e2771797bf4f admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:36.178058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1126/2276] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:36 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:41:36.179459 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e447238-9cc0-49b4-b493-99f7640cb359 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:36.196531 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e447238-9cc0-49b4-b493-99f7640cb359 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:36.269129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb989ea-3cca-4890-a339-947677a927cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:36.269805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb989ea-3cca-4890-a339-947677a927cc admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:36.270122 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb989ea-3cca-4890-a339-947677a927cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:36.270400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb989ea-3cca-4890-a339-947677a927cc admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:36.271101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb989ea-3cca-4890-a339-947677a927cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:36.273027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb989ea-3cca-4890-a339-947677a927cc admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:36.301362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1151/2277] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:36 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:36.302668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f847ad3-3789-442b-8971-e2771797bf4f admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:36.313877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f847ad3-3789-442b-8971-e2771797bf4f admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:36.319397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542a971-31b8-4e58-bc0d-8aa017fed725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:36.320261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542a971-31b8-4e58-bc0d-8aa017fed725 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:36.320707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542a971-31b8-4e58-bc0d-8aa017fed725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:36.321145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542a971-31b8-4e58-bc0d-8aa017fed725 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:36.322054 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542a971-31b8-4e58-bc0d-8aa017fed725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:36.323818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542a971-31b8-4e58-bc0d-8aa017fed725 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:36.337213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1127/2278] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:36.340897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb989ea-3cca-4890-a339-947677a927cc admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:36.357524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb989ea-3cca-4890-a339-947677a927cc admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:36.362733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa0e15-ef0a-4291-96a3-7877107a00da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:36.363625 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa0e15-ef0a-4291-96a3-7877107a00da admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:36.363913 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa0e15-ef0a-4291-96a3-7877107a00da admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:36.364195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa0e15-ef0a-4291-96a3-7877107a00da admin admin] PATH_INFO: `/v3/projects/7c2b395f43354844af9258dc87ead1b3/users/41bd784c91884859ba9afd49460a1be8/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:36.366732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:36.386438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa0e15-ef0a-4291-96a3-7877107a00da admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c2b395f43354844af9258dc87ead1b3, user_id=41bd784c91884859ba9afd49460a1be8, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:36.389766 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa0e15-ef0a-4291-96a3-7877107a00da admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:36.409253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542a971-31b8-4e58-bc0d-8aa017fed725 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:36.410343 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1152/2279] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:36 2022] PUT /identity/v3/projects/7c2b395f43354844af9258dc87ead1b3/users/41bd784c91884859ba9afd49460a1be8/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:36.422317 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542a971-31b8-4e58-bc0d-8aa017fed725 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:36.485593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c20c7-a506-4b9c-9b6e-6e56892795d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:36.486126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c20c7-a506-4b9c-9b6e-6e56892795d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:36.486413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c20c7-a506-4b9c-9b6e-6e56892795d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:36.486705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c20c7-a506-4b9c-9b6e-6e56892795d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:36.487519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c20c7-a506-4b9c-9b6e-6e56892795d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:36.489128 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c20c7-a506-4b9c-9b6e-6e56892795d8 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:36.500892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1128/2280] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:36.505008 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa0e15-ef0a-4291-96a3-7877107a00da admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:36.515895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa0e15-ef0a-4291-96a3-7877107a00da admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:36.520832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85e13a-38c9-4619-95a7-0cfaf3b49997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:36.521724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85e13a-38c9-4619-95a7-0cfaf3b49997 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:36.522059 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85e13a-38c9-4619-95a7-0cfaf3b49997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:36.522341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85e13a-38c9-4619-95a7-0cfaf3b49997 admin admin] PATH_INFO: `/v3/projects/7c2b395f43354844af9258dc87ead1b3/users/41bd784c91884859ba9afd49460a1be8/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:36.524375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85e13a-38c9-4619-95a7-0cfaf3b49997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c2b395f43354844af9258dc87ead1b3, user_id=41bd784c91884859ba9afd49460a1be8, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:36.527062 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85e13a-38c9-4619-95a7-0cfaf3b49997 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:36.548668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1153/2281] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:36 2022] PUT /identity/v3/projects/7c2b395f43354844af9258dc87ead1b3/users/41bd784c91884859ba9afd49460a1be8/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:36.559166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f657e-07eb-41f3-9f20-7c99abc47f4b None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:36.559456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f657e-07eb-41f3-9f20-7c99abc47f4b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:36.559735 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f657e-07eb-41f3-9f20-7c99abc47f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:36.589548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-864f657e-07eb-41f3-9f20-7c99abc47f4b None None] MFA Rules not processed for user `41bd784c91884859ba9afd49460a1be8`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:36.591378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-864f657e-07eb-41f3-9f20-7c99abc47f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:36.632648 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:36.664366 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1129/2282] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:36.678433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85e13a-38c9-4619-95a7-0cfaf3b49997 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:36.680107 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fc85e13a-38c9-4619-95a7-0cfaf3b49997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:36.738875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85e13a-38c9-4619-95a7-0cfaf3b49997 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:36.742673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd838582-1a7d-4ee0-834c-16116424c43f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:36.743389 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd838582-1a7d-4ee0-834c-16116424c43f service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:36.743739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd838582-1a7d-4ee0-834c-16116424c43f service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:36.744081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd838582-1a7d-4ee0-834c-16116424c43f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:36.753192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd838582-1a7d-4ee0-834c-16116424c43f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:36.754695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd838582-1a7d-4ee0-834c-16116424c43f service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:36.764167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1154/2283] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:36.856815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f657e-07eb-41f3-9f20-7c99abc47f4b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:36.867156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f657e-07eb-41f3-9f20-7c99abc47f4b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:36.871715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88202c0-07b4-4f2a-b656-2b6bcb272a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:36.872328 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88202c0-07b4-4f2a-b656-2b6bcb272a7e service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:36.872611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88202c0-07b4-4f2a-b656-2b6bcb272a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:36.872891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88202c0-07b4-4f2a-b656-2b6bcb272a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:36.938903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88202c0-07b4-4f2a-b656-2b6bcb272a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:36.941633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88202c0-07b4-4f2a-b656-2b6bcb272a7e service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:36.955765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1130/2284] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:41:37.221503 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd838582-1a7d-4ee0-834c-16116424c43f service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:37.282159 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd838582-1a7d-4ee0-834c-16116424c43f service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:37.286715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:37.287577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:37.287856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:37.288156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] PATH_INFO: `/v3/users/d3d5e2c2e13a4d6297e0dc263e1404a3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:37.289348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3d5e2c2e13a4d6297e0dc263e1404a3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:37.291303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:37.337459 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3d5e2c2e13a4d6297e0dc263e1404a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:37.344897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3d5e2c2e13a4d6297e0dc263e1404a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:37.345601 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3d5e2c2e13a4d6297e0dc263e1404a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:37.352742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3d5e2c2e13a4d6297e0dc263e1404a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:37.353378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3d5e2c2e13a4d6297e0dc263e1404a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:37.361271 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:37.372590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1155/2285] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:37 2022] DELETE /identity/v3/users/d3d5e2c2e13a4d6297e0dc263e1404a3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:37.377167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac013596-812a-4700-8160-40a5f58df895 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:37.377459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac013596-812a-4700-8160-40a5f58df895 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:37.377742 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac013596-812a-4700-8160-40a5f58df895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:37.397055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-ac013596-812a-4700-8160-40a5f58df895 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:37.398311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ac013596-812a-4700-8160-40a5f58df895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:37.457130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1131/2286] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:41:37.467143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:37.468620 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:37.533433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d2279-8622-4824-8528-b91bdc487e8a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:37.537350 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550c110-b07a-4da0-b691-07e2cbdfacfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:37.538016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550c110-b07a-4da0-b691-07e2cbdfacfd service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:37.538302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550c110-b07a-4da0-b691-07e2cbdfacfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:37.538577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550c110-b07a-4da0-b691-07e2cbdfacfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:37.549333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550c110-b07a-4da0-b691-07e2cbdfacfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:37.550591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550c110-b07a-4da0-b691-07e2cbdfacfd service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:37.566454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1156/2287] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:37.858698 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac013596-812a-4700-8160-40a5f58df895 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:37.860496 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ac013596-812a-4700-8160-40a5f58df895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:37.874289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac013596-812a-4700-8160-40a5f58df895 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:37.879706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:37.880590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:37.880876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:37.881152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] PATH_INFO: `/v3/projects/df5a90f4ca324da083b7bd2fe5941f91` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:37.882418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df5a90f4ca324da083b7bd2fe5941f91)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:37.884337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:37.932130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df5a90f4ca324da083b7bd2fe5941f91', 'request_id': 'req-255682cf-7f90-4e69-865e-08764d0a2862'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:37.939063 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df5a90f4ca324da083b7bd2fe5941f91', 'request_id': 'req-255682cf-7f90-4e69-865e-08764d0a2862'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:37.944563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df5a90f4ca324da083b7bd2fe5941f91', 'request_id': 'req-255682cf-7f90-4e69-865e-08764d0a2862'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:37.952862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:37.958029 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] The token cache is being invalidate because project df5a90f4ca324da083b7bd2fe5941f91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:37.958576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:37.959675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:37.961606 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1132/2288] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:37 2022] DELETE /identity/v3/projects/df5a90f4ca324da083b7bd2fe5941f91 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:37.966723 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550c110-b07a-4da0-b691-07e2cbdfacfd service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:37.976854 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550c110-b07a-4da0-b691-07e2cbdfacfd service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:38.035597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cd535-86e4-47d7-acce-cba6cf0e6739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:38.036458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cd535-86e4-47d7-acce-cba6cf0e6739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:38.036760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cd535-86e4-47d7-acce-cba6cf0e6739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:38.037035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cd535-86e4-47d7-acce-cba6cf0e6739 admin admin] PATH_INFO: `/v3/users/41bd784c91884859ba9afd49460a1be8` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:38.038247 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cd535-86e4-47d7-acce-cba6cf0e6739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41bd784c91884859ba9afd49460a1be8)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:38.040189 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cd535-86e4-47d7-acce-cba6cf0e6739 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:38.086443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ba5cd535-86e4-47d7-acce-cba6cf0e6739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41bd784c91884859ba9afd49460a1be8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:38.094985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ba5cd535-86e4-47d7-acce-cba6cf0e6739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41bd784c91884859ba9afd49460a1be8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:38.095628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ba5cd535-86e4-47d7-acce-cba6cf0e6739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41bd784c91884859ba9afd49460a1be8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:38.102413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ba5cd535-86e4-47d7-acce-cba6cf0e6739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41bd784c91884859ba9afd49460a1be8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:38.103044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ba5cd535-86e4-47d7-acce-cba6cf0e6739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41bd784c91884859ba9afd49460a1be8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:38.120886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1157/2289] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:37 2022] DELETE /identity/v3/users/41bd784c91884859ba9afd49460a1be8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:38.433991 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:38.435447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:38.450266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255682cf-7f90-4e69-865e-08764d0a2862 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:38.514544 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fcc08-b027-4cb1-a054-6b296c687cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:38.515308 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fcc08-b027-4cb1-a054-6b296c687cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:38.515594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fcc08-b027-4cb1-a054-6b296c687cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:38.515872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fcc08-b027-4cb1-a054-6b296c687cee admin admin] PATH_INFO: `/v3/projects/7c2b395f43354844af9258dc87ead1b3` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:38.517139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755fcc08-b027-4cb1-a054-6b296c687cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c2b395f43354844af9258dc87ead1b3)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:38.519247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755fcc08-b027-4cb1-a054-6b296c687cee admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:38.565450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-755fcc08-b027-4cb1-a054-6b296c687cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c2b395f43354844af9258dc87ead1b3', 'request_id': 'req-755fcc08-b027-4cb1-a054-6b296c687cee'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:38.571868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-755fcc08-b027-4cb1-a054-6b296c687cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c2b395f43354844af9258dc87ead1b3', 'request_id': 'req-755fcc08-b027-4cb1-a054-6b296c687cee'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:38.578111 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-755fcc08-b027-4cb1-a054-6b296c687cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c2b395f43354844af9258dc87ead1b3', 'request_id': 'req-755fcc08-b027-4cb1-a054-6b296c687cee'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:38.592830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-755fcc08-b027-4cb1-a054-6b296c687cee admin admin] The token cache is being invalidate because project 7c2b395f43354844af9258dc87ead1b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:38.593114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-755fcc08-b027-4cb1-a054-6b296c687cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:38.593850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-755fcc08-b027-4cb1-a054-6b296c687cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:38.595428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1133/2290] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:38 2022] DELETE /identity/v3/projects/7c2b395f43354844af9258dc87ead1b3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:38.600809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979ae59-cd59-4406-a869-003ca2f5a534 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:38.601315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979ae59-cd59-4406-a869-003ca2f5a534 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:38.601691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979ae59-cd59-4406-a869-003ca2f5a534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:38.623696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-8979ae59-cd59-4406-a869-003ca2f5a534 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:38.625319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8979ae59-cd59-4406-a869-003ca2f5a534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:38.634280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:38.724970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1158/2291] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:38.733657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68827594-7b11-489d-8a6b-945b3bb2932d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:38.734209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68827594-7b11-489d-8a6b-945b3bb2932d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:38.734781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68827594-7b11-489d-8a6b-945b3bb2932d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:38.768298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-68827594-7b11-489d-8a6b-945b3bb2932d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:38.770259 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-68827594-7b11-489d-8a6b-945b3bb2932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:38.780510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1134/2292] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:38.785038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979ae59-cd59-4406-a869-003ca2f5a534 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:38.797379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979ae59-cd59-4406-a869-003ca2f5a534 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:38.802379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee48a9f-8e8c-49cd-91f3-6b93652d3860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:38.803214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee48a9f-8e8c-49cd-91f3-6b93652d3860 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:38.803652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee48a9f-8e8c-49cd-91f3-6b93652d3860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:38.803922 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee48a9f-8e8c-49cd-91f3-6b93652d3860 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:38.807232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee48a9f-8e8c-49cd-91f3-6b93652d3860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:38.807232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee48a9f-8e8c-49cd-91f3-6b93652d3860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:38.808844 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee48a9f-8e8c-49cd-91f3-6b93652d3860 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:38.830614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1159/2293] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:41:38.835076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68827594-7b11-489d-8a6b-945b3bb2932d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:38.846773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68827594-7b11-489d-8a6b-945b3bb2932d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:38.851589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d932c81-efbd-4717-9960-94a51c402019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:38.852340 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d932c81-efbd-4717-9960-94a51c402019 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:38.852624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d932c81-efbd-4717-9960-94a51c402019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:38.852899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d932c81-efbd-4717-9960-94a51c402019 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:38.853641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d932c81-efbd-4717-9960-94a51c402019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:38.855622 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d932c81-efbd-4717-9960-94a51c402019 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:38.892232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1135/2294] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:38 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:38.893441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee48a9f-8e8c-49cd-91f3-6b93652d3860 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:38.903906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee48a9f-8e8c-49cd-91f3-6b93652d3860 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:38.977949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9238c1-d003-4636-9d7f-c7d7267736a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:38.978902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9238c1-d003-4636-9d7f-c7d7267736a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:38.979292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9238c1-d003-4636-9d7f-c7d7267736a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:38.979590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9238c1-d003-4636-9d7f-c7d7267736a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:38.980192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9238c1-d003-4636-9d7f-c7d7267736a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:38.982180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9238c1-d003-4636-9d7f-c7d7267736a1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:39.009900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1160/2295] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:38 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:41:39.014661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d932c81-efbd-4717-9960-94a51c402019 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:39.028702 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d932c81-efbd-4717-9960-94a51c402019 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:39.034740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c272b0-eb22-4c6b-b6a1-858061bfaf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:39.035524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c272b0-eb22-4c6b-b6a1-858061bfaf5f admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:39.035821 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c272b0-eb22-4c6b-b6a1-858061bfaf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:39.036134 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c272b0-eb22-4c6b-b6a1-858061bfaf5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:39.037072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c272b0-eb22-4c6b-b6a1-858061bfaf5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:39.038971 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c272b0-eb22-4c6b-b6a1-858061bfaf5f admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:39.053960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1136/2296] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:39.057371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9238c1-d003-4636-9d7f-c7d7267736a1 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:39.070483 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9238c1-d003-4636-9d7f-c7d7267736a1 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:39.077194 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786ea54-e214-4aff-9d73-1ad36d0bba86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:39.082123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786ea54-e214-4aff-9d73-1ad36d0bba86 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:39.082123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786ea54-e214-4aff-9d73-1ad36d0bba86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:39.082123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786ea54-e214-4aff-9d73-1ad36d0bba86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:39.082123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786ea54-e214-4aff-9d73-1ad36d0bba86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:39.082123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786ea54-e214-4aff-9d73-1ad36d0bba86 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:39.096817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1161/2297] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:39.101435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c272b0-eb22-4c6b-b6a1-858061bfaf5f admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:39.114878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c272b0-eb22-4c6b-b6a1-858061bfaf5f admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:39.121376 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1c742-ab0e-45a7-b804-f9e05dc02668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:39.122924 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1c742-ab0e-45a7-b804-f9e05dc02668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:39.123208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1c742-ab0e-45a7-b804-f9e05dc02668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:39.123483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1c742-ab0e-45a7-b804-f9e05dc02668 admin admin] PATH_INFO: `/v3/projects/f2e9e9773b6f4a61a88ef01709bc90f0/users/442c19c758054983b5468dea0546a063/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:39.147160 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1c742-ab0e-45a7-b804-f9e05dc02668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2e9e9773b6f4a61a88ef01709bc90f0, user_id=442c19c758054983b5468dea0546a063, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:39.150948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1c742-ab0e-45a7-b804-f9e05dc02668 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:39.167034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1137/2298] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:39 2022] PUT /identity/v3/projects/f2e9e9773b6f4a61a88ef01709bc90f0/users/442c19c758054983b5468dea0546a063/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:39.178018 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b127ed-dcdc-4476-a97e-be8e1aed0aac None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:39.178302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b127ed-dcdc-4476-a97e-be8e1aed0aac None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:39.178695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b127ed-dcdc-4476-a97e-be8e1aed0aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:39.216612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-67b127ed-dcdc-4476-a97e-be8e1aed0aac None None] MFA Rules not processed for user `442c19c758054983b5468dea0546a063`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:39.217817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-67b127ed-dcdc-4476-a97e-be8e1aed0aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:39.284486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1162/2299] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:39.298175 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1c742-ab0e-45a7-b804-f9e05dc02668 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:39.299956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5ea1c742-ab0e-45a7-b804-f9e05dc02668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:39.327336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:39.359163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1c742-ab0e-45a7-b804-f9e05dc02668 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:39.363117 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792b8f2-dd48-4207-b4ba-f76b95d182f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:39.363886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792b8f2-dd48-4207-b4ba-f76b95d182f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:39.364119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792b8f2-dd48-4207-b4ba-f76b95d182f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:39.364486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792b8f2-dd48-4207-b4ba-f76b95d182f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:39.373944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792b8f2-dd48-4207-b4ba-f76b95d182f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:39.375399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792b8f2-dd48-4207-b4ba-f76b95d182f4 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:39.385262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1138/2300] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:39.406566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b127ed-dcdc-4476-a97e-be8e1aed0aac None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:39.421680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:39.484706 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b127ed-dcdc-4476-a97e-be8e1aed0aac None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:39.489363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:39.490291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:39.490577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:39.490851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] PATH_INFO: `/v3/users/442c19c758054983b5468dea0546a063` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:39.492065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=442c19c758054983b5468dea0546a063)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:39.494038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:39.545609 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '442c19c758054983b5468dea0546a063'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:39.553737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '442c19c758054983b5468dea0546a063'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:39.554455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '442c19c758054983b5468dea0546a063'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:39.562027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '442c19c758054983b5468dea0546a063'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:39.562695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '442c19c758054983b5468dea0546a063'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:39.581578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1163/2301] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:39 2022] DELETE /identity/v3/users/442c19c758054983b5468dea0546a063 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:39.586698 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c2baa-c787-47b3-bc3f-563eda500542 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:39.587057 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c2baa-c787-47b3-bc3f-563eda500542 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:39.587416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c2baa-c787-47b3-bc3f-563eda500542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:39.607953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-7b3c2baa-c787-47b3-bc3f-563eda500542 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:39.609234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7b3c2baa-c787-47b3-bc3f-563eda500542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:39.672416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1139/2302] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:39.689624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:39.691290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:39.753927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74130c13-ba38-4a71-87fb-7139715afc53 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:39.757655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bc2e0-a68b-4693-acb9-994f654623e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:39.758416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bc2e0-a68b-4693-acb9-994f654623e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:39.758771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bc2e0-a68b-4693-acb9-994f654623e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:39.759116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bc2e0-a68b-4693-acb9-994f654623e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:39.769319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bc2e0-a68b-4693-acb9-994f654623e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:39.770787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bc2e0-a68b-4693-acb9-994f654623e8 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:39.781016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1164/2303] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:40.145342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c2baa-c787-47b3-bc3f-563eda500542 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:40.147604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7b3c2baa-c787-47b3-bc3f-563eda500542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:40.161671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c2baa-c787-47b3-bc3f-563eda500542 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:40.167718 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9800b9a-2bbf-4ea4-be44-0b473d292b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:40.169418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9800b9a-2bbf-4ea4-be44-0b473d292b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:40.170814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9800b9a-2bbf-4ea4-be44-0b473d292b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:40.171331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9800b9a-2bbf-4ea4-be44-0b473d292b61 admin admin] PATH_INFO: `/v3/projects/f2e9e9773b6f4a61a88ef01709bc90f0` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:40.176980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9800b9a-2bbf-4ea4-be44-0b473d292b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2e9e9773b6f4a61a88ef01709bc90f0)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:40.189403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9800b9a-2bbf-4ea4-be44-0b473d292b61 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:40.308089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c9800b9a-2bbf-4ea4-be44-0b473d292b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2e9e9773b6f4a61a88ef01709bc90f0', 'request_id': 'req-c9800b9a-2bbf-4ea4-be44-0b473d292b61'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:40.315361 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c9800b9a-2bbf-4ea4-be44-0b473d292b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2e9e9773b6f4a61a88ef01709bc90f0', 'request_id': 'req-c9800b9a-2bbf-4ea4-be44-0b473d292b61'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:40.321401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c9800b9a-2bbf-4ea4-be44-0b473d292b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2e9e9773b6f4a61a88ef01709bc90f0', 'request_id': 'req-c9800b9a-2bbf-4ea4-be44-0b473d292b61'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:40.343113 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c9800b9a-2bbf-4ea4-be44-0b473d292b61 admin admin] The token cache is being invalidate because project f2e9e9773b6f4a61a88ef01709bc90f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:40.343404 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c9800b9a-2bbf-4ea4-be44-0b473d292b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:40.344066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c9800b9a-2bbf-4ea4-be44-0b473d292b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:40.346039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1140/2304] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:40 2022] DELETE /identity/v3/projects/f2e9e9773b6f4a61a88ef01709bc90f0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:40.351591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93294a4-1041-4552-9bee-c98446ecfe99 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:40.351884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93294a4-1041-4552-9bee-c98446ecfe99 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:40.352168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93294a4-1041-4552-9bee-c98446ecfe99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:40.373763 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b93294a4-1041-4552-9bee-c98446ecfe99 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:40.375337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b93294a4-1041-4552-9bee-c98446ecfe99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:40.404160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:40.451601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b25b4-b10d-47b9-85e0-e115111aeeea None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:40.452436 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b25b4-b10d-47b9-85e0-e115111aeeea None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:40.452436 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b25b4-b10d-47b9-85e0-e115111aeeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:40.458311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1165/2305] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:41:40.478511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-013b25b4-b10d-47b9-85e0-e115111aeeea None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:40.480069 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-013b25b4-b10d-47b9-85e0-e115111aeeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:40.494295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1141/2306] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:40.499192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93294a4-1041-4552-9bee-c98446ecfe99 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:40.510612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93294a4-1041-4552-9bee-c98446ecfe99 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:40.518222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2b883-a4a5-48b3-ad1d-603e2311e108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:40.518886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2b883-a4a5-48b3-ad1d-603e2311e108 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:40.519190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2b883-a4a5-48b3-ad1d-603e2311e108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:40.519460 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2b883-a4a5-48b3-ad1d-603e2311e108 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:40.520203 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2b883-a4a5-48b3-ad1d-603e2311e108 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:41:40.520726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2b883-a4a5-48b3-ad1d-603e2311e108 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:40.522166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2b883-a4a5-48b3-ad1d-603e2311e108 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:40.540600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1166/2307] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:41:40.551298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b25b4-b10d-47b9-85e0-e115111aeeea None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:40.561825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b25b4-b10d-47b9-85e0-e115111aeeea None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:40.568068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757358e0-f4ed-4d06-870a-9389ba2a5404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:40.568738 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757358e0-f4ed-4d06-870a-9389ba2a5404 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:40.569018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757358e0-f4ed-4d06-870a-9389ba2a5404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:40.569291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757358e0-f4ed-4d06-870a-9389ba2a5404 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:40.570090 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757358e0-f4ed-4d06-870a-9389ba2a5404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:40.572663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757358e0-f4ed-4d06-870a-9389ba2a5404 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:40.633582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1142/2308] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:41:40 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:40.634748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2b883-a4a5-48b3-ad1d-603e2311e108 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:40.647199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2b883-a4a5-48b3-ad1d-603e2311e108 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:40.854296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907c692-a638-49fb-a618-f5e764dbefdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:40.854993 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907c692-a638-49fb-a618-f5e764dbefdd admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:40.855282 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907c692-a638-49fb-a618-f5e764dbefdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:40.855585 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907c692-a638-49fb-a618-f5e764dbefdd admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:40.856359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907c692-a638-49fb-a618-f5e764dbefdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:40.858350 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907c692-a638-49fb-a618-f5e764dbefdd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:40.881737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1167/2309] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:41:40 2022] POST /identity/v3/users => generated 291 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:41:40.886452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757358e0-f4ed-4d06-870a-9389ba2a5404 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:40.898934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757358e0-f4ed-4d06-870a-9389ba2a5404 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:40.904222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c0e6e-589d-4641-8d27-ee7ccbe4885a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:40.904882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c0e6e-589d-4641-8d27-ee7ccbe4885a admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:40.905165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c0e6e-589d-4641-8d27-ee7ccbe4885a admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:40.905443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c0e6e-589d-4641-8d27-ee7ccbe4885a admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:40.906198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c0e6e-589d-4641-8d27-ee7ccbe4885a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:40.907785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c0e6e-589d-4641-8d27-ee7ccbe4885a admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:40.917842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1143/2310] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:41:40.921817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907c692-a638-49fb-a618-f5e764dbefdd admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:40.931809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907c692-a638-49fb-a618-f5e764dbefdd admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:40.937583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d81dd-cc9b-43d0-a472-3aeab5b16520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:40.937583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d81dd-cc9b-43d0-a472-3aeab5b16520 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:40.937583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d81dd-cc9b-43d0-a472-3aeab5b16520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:40.937865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d81dd-cc9b-43d0-a472-3aeab5b16520 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:40.938430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d81dd-cc9b-43d0-a472-3aeab5b16520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:40.940019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d81dd-cc9b-43d0-a472-3aeab5b16520 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:40.950874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1168/2311] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:41:40.955066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c0e6e-589d-4641-8d27-ee7ccbe4885a admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:40.974188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c0e6e-589d-4641-8d27-ee7ccbe4885a admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:40.978675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a371586-b59f-4018-9cf2-3b9590719747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:40.979555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a371586-b59f-4018-9cf2-3b9590719747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:40.979834 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a371586-b59f-4018-9cf2-3b9590719747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:40.980104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a371586-b59f-4018-9cf2-3b9590719747 admin admin] PATH_INFO: `/v3/projects/cf5117b767c54f7ca763c72d1852834e/users/30538f9a7b5d44a88c66d42a4ef4533f/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:40.989201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:40.998418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a371586-b59f-4018-9cf2-3b9590719747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5117b767c54f7ca763c72d1852834e, user_id=30538f9a7b5d44a88c66d42a4ef4533f, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:41.001321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a371586-b59f-4018-9cf2-3b9590719747 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:41.014872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1144/2312] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:41:40 2022] PUT /identity/v3/projects/cf5117b767c54f7ca763c72d1852834e/users/30538f9a7b5d44a88c66d42a4ef4533f/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:41:41.027150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d87ab-330d-486d-88fb-eae92d7e79dd None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:41.027468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d87ab-330d-486d-88fb-eae92d7e79dd None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:41.027836 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d87ab-330d-486d-88fb-eae92d7e79dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:41.055141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-251d87ab-330d-486d-88fb-eae92d7e79dd None None] MFA Rules not processed for user `30538f9a7b5d44a88c66d42a4ef4533f`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:41.056355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-251d87ab-330d-486d-88fb-eae92d7e79dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:41.125000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1169/2313] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:41.132636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a371586-b59f-4018-9cf2-3b9590719747 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:41.134381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0a371586-b59f-4018-9cf2-3b9590719747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:41.196802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a371586-b59f-4018-9cf2-3b9590719747 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:41.201101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e0dcd-e82f-46f2-b4be-4f2a456e9221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:41.201753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e0dcd-e82f-46f2-b4be-4f2a456e9221 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:41.202069 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e0dcd-e82f-46f2-b4be-4f2a456e9221 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:41.202345 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e0dcd-e82f-46f2-b4be-4f2a456e9221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:41.210984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e0dcd-e82f-46f2-b4be-4f2a456e9221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:41.212399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e0dcd-e82f-46f2-b4be-4f2a456e9221 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:41.221446 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1145/2314] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:45.900718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d87ab-330d-486d-88fb-eae92d7e79dd None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:45.929490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:45.964732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d87ab-330d-486d-88fb-eae92d7e79dd None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:45.969969 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:45.970830 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:45.971102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:45.971369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] PATH_INFO: `/v3/users/30538f9a7b5d44a88c66d42a4ef4533f` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:45.972722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30538f9a7b5d44a88c66d42a4ef4533f)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:45.974708 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:46.028339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30538f9a7b5d44a88c66d42a4ef4533f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.036684 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30538f9a7b5d44a88c66d42a4ef4533f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.037373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30538f9a7b5d44a88c66d42a4ef4533f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.045226 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30538f9a7b5d44a88c66d42a4ef4533f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.045988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30538f9a7b5d44a88c66d42a4ef4533f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.063587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1170/2315] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:45 2022] DELETE /identity/v3/users/30538f9a7b5d44a88c66d42a4ef4533f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:46.068218 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f239f4a-6f6d-41eb-8f14-6bdf43d20e81 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:46.068809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f239f4a-6f6d-41eb-8f14-6bdf43d20e81 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:46.068912 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f239f4a-6f6d-41eb-8f14-6bdf43d20e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:46.090676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-1f239f4a-6f6d-41eb-8f14-6bdf43d20e81 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:46.091914 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-1f239f4a-6f6d-41eb-8f14-6bdf43d20e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:46.156882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1146/2316] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:41:46.166740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:46.168296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:46.229175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c9db3-a471-4557-ad3a-0b7faadda5c7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:46.232728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16185f-145d-49a1-804a-fd0644e272d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:46.233365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16185f-145d-49a1-804a-fd0644e272d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:46.233645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16185f-145d-49a1-804a-fd0644e272d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:46.233937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16185f-145d-49a1-804a-fd0644e272d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:46.243680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16185f-145d-49a1-804a-fd0644e272d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:46.245417 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16185f-145d-49a1-804a-fd0644e272d5 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:46.256197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1171/2317] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:46.524051 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f239f4a-6f6d-41eb-8f14-6bdf43d20e81 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:46.526021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-1f239f4a-6f6d-41eb-8f14-6bdf43d20e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:46.539378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f239f4a-6f6d-41eb-8f14-6bdf43d20e81 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:46.544553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364205b-8cf4-4137-9601-8201fe1ecfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:46.545955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364205b-8cf4-4137-9601-8201fe1ecfa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:46.546248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364205b-8cf4-4137-9601-8201fe1ecfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:46.546526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364205b-8cf4-4137-9601-8201fe1ecfa7 admin admin] PATH_INFO: `/v3/projects/cf5117b767c54f7ca763c72d1852834e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:46.547745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364205b-8cf4-4137-9601-8201fe1ecfa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf5117b767c54f7ca763c72d1852834e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:46.549745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364205b-8cf4-4137-9601-8201fe1ecfa7 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:46.608180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2364205b-8cf4-4137-9601-8201fe1ecfa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf5117b767c54f7ca763c72d1852834e', 'request_id': 'req-2364205b-8cf4-4137-9601-8201fe1ecfa7'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.615379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2364205b-8cf4-4137-9601-8201fe1ecfa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf5117b767c54f7ca763c72d1852834e', 'request_id': 'req-2364205b-8cf4-4137-9601-8201fe1ecfa7'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.622475 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2364205b-8cf4-4137-9601-8201fe1ecfa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf5117b767c54f7ca763c72d1852834e', 'request_id': 'req-2364205b-8cf4-4137-9601-8201fe1ecfa7'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.636453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2364205b-8cf4-4137-9601-8201fe1ecfa7 admin admin] The token cache is being invalidate because project cf5117b767c54f7ca763c72d1852834e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:46.636745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2364205b-8cf4-4137-9601-8201fe1ecfa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.637413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2364205b-8cf4-4137-9601-8201fe1ecfa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.639080 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1147/2318] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:46 2022] DELETE /identity/v3/projects/cf5117b767c54f7ca763c72d1852834e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:46.801709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16185f-145d-49a1-804a-fd0644e272d5 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:46.803193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cb16185f-145d-49a1-804a-fd0644e272d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:46.818381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16185f-145d-49a1-804a-fd0644e272d5 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:46.868743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:46.886210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:46.887126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:46.890370 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:46.890370 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] PATH_INFO: `/v3/users/ab5e3c513d1b458d9620e575265ddb83` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:46.891326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab5e3c513d1b458d9620e575265ddb83)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:46.893343 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:46.946972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab5e3c513d1b458d9620e575265ddb83'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.953963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab5e3c513d1b458d9620e575265ddb83'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.954575 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab5e3c513d1b458d9620e575265ddb83'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.961322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab5e3c513d1b458d9620e575265ddb83'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.961959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab5e3c513d1b458d9620e575265ddb83'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:46.981333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1172/2319] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:41:46 2022] DELETE /identity/v3/users/ab5e3c513d1b458d9620e575265ddb83 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:46.986980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6683f-0a7b-410c-830a-19095a2e3e8e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:46.987300 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6683f-0a7b-410c-830a-19095a2e3e8e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:46.987611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6683f-0a7b-410c-830a-19095a2e3e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:47.007278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-97d6683f-0a7b-410c-830a-19095a2e3e8e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:41:47.008508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-97d6683f-0a7b-410c-830a-19095a2e3e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:47.017069 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:47.078593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1148/2320] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:41:47.085250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:47.086756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:47.145588 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2df7d-0260-4f1e-849f-6547b9980bf1 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:47.149268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623c23e-9b9c-41fc-9ff9-8b8118a983c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:47.149936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623c23e-9b9c-41fc-9ff9-8b8118a983c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:47.150237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623c23e-9b9c-41fc-9ff9-8b8118a983c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:47.150515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623c23e-9b9c-41fc-9ff9-8b8118a983c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:47.160306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623c23e-9b9c-41fc-9ff9-8b8118a983c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:47.161758 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623c23e-9b9c-41fc-9ff9-8b8118a983c2 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:47.171753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1173/2321] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:41:47.447627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6683f-0a7b-410c-830a-19095a2e3e8e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:41:47.449483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-97d6683f-0a7b-410c-830a-19095a2e3e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:41:47.462104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6683f-0a7b-410c-830a-19095a2e3e8e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:41:47.466663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02215a65-0461-4f08-b46b-551e09e8c113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:41:47.468123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02215a65-0461-4f08-b46b-551e09e8c113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:41:47.468405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02215a65-0461-4f08-b46b-551e09e8c113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:41:47.468810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02215a65-0461-4f08-b46b-551e09e8c113 admin admin] PATH_INFO: `/v3/projects/d1530a1e5bf24475bb0bf9725401c09a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:41:47.470024 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02215a65-0461-4f08-b46b-551e09e8c113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1530a1e5bf24475bb0bf9725401c09a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:41:47.472739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02215a65-0461-4f08-b46b-551e09e8c113 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:41:47.520360 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-02215a65-0461-4f08-b46b-551e09e8c113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1530a1e5bf24475bb0bf9725401c09a', 'request_id': 'req-02215a65-0461-4f08-b46b-551e09e8c113'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:47.526488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-02215a65-0461-4f08-b46b-551e09e8c113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1530a1e5bf24475bb0bf9725401c09a', 'request_id': 'req-02215a65-0461-4f08-b46b-551e09e8c113'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:47.533522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-02215a65-0461-4f08-b46b-551e09e8c113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1530a1e5bf24475bb0bf9725401c09a', 'request_id': 'req-02215a65-0461-4f08-b46b-551e09e8c113'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:47.545669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-02215a65-0461-4f08-b46b-551e09e8c113 admin admin] The token cache is being invalidate because project d1530a1e5bf24475bb0bf9725401c09a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:41:47.545976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-02215a65-0461-4f08-b46b-551e09e8c113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:47.546616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-02215a65-0461-4f08-b46b-551e09e8c113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:41:47.548136 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1149/2322] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:41:47 2022] DELETE /identity/v3/projects/d1530a1e5bf24475bb0bf9725401c09a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:41:57.166987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:41:57.542632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:42:15.323370 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623c23e-9b9c-41fc-9ff9-8b8118a983c2 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:15.324754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a623c23e-9b9c-41fc-9ff9-8b8118a983c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:15.337788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623c23e-9b9c-41fc-9ff9-8b8118a983c2 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:15.394926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:15.395775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:15.396057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:15.396334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] PATH_INFO: `/v3/users/820f31be79d240e5bac59fdd54748fbf` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:15.397519 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] RBAC: Authorizing `identity:delete_user(user_id=820f31be79d240e5bac59fdd54748fbf)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:15.399485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:15.445752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '820f31be79d240e5bac59fdd54748fbf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:15.454907 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '820f31be79d240e5bac59fdd54748fbf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:15.455615 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '820f31be79d240e5bac59fdd54748fbf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:15.464158 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '820f31be79d240e5bac59fdd54748fbf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:15.465108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '820f31be79d240e5bac59fdd54748fbf'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:15.489965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1174/2323] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:42:15 2022] DELETE /identity/v3/users/820f31be79d240e5bac59fdd54748fbf => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:42:15.495781 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116f3b3-0cbb-45dc-a4f8-fb765cccd40b None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:15.496293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116f3b3-0cbb-45dc-a4f8-fb765cccd40b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:15.496377 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116f3b3-0cbb-45dc-a4f8-fb765cccd40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:15.520465 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-1116f3b3-0cbb-45dc-a4f8-fb765cccd40b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:15.522278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-1116f3b3-0cbb-45dc-a4f8-fb765cccd40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:15.585747 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1150/2324] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:42:15.596284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:15.597682 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:15.656613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af3059-dbab-4d80-95fe-b69a0b84659d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:15.660251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f8480-4cf1-42f6-9063-4ba7fd2e03ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:15.660952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f8480-4cf1-42f6-9063-4ba7fd2e03ba service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:15.661156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f8480-4cf1-42f6-9063-4ba7fd2e03ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:15.661429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f8480-4cf1-42f6-9063-4ba7fd2e03ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:15.671149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f8480-4cf1-42f6-9063-4ba7fd2e03ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:15.672590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f8480-4cf1-42f6-9063-4ba7fd2e03ba service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:15.682526 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1175/2325] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:42:15.980150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116f3b3-0cbb-45dc-a4f8-fb765cccd40b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:15.981585 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-1116f3b3-0cbb-45dc-a4f8-fb765cccd40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:15.994946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116f3b3-0cbb-45dc-a4f8-fb765cccd40b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:16.006123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47baebc-a61d-430f-aa39-e258b23f0e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:16.007010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47baebc-a61d-430f-aa39-e258b23f0e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:16.007287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47baebc-a61d-430f-aa39-e258b23f0e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:16.007559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47baebc-a61d-430f-aa39-e258b23f0e49 admin admin] PATH_INFO: `/v3/projects/147e44123a2b40caa56a9871ca92b3e9` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:16.008820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47baebc-a61d-430f-aa39-e258b23f0e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=147e44123a2b40caa56a9871ca92b3e9)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:16.010958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47baebc-a61d-430f-aa39-e258b23f0e49 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:16.062013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f47baebc-a61d-430f-aa39-e258b23f0e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147e44123a2b40caa56a9871ca92b3e9', 'request_id': 'req-f47baebc-a61d-430f-aa39-e258b23f0e49'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:16.069683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f47baebc-a61d-430f-aa39-e258b23f0e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147e44123a2b40caa56a9871ca92b3e9', 'request_id': 'req-f47baebc-a61d-430f-aa39-e258b23f0e49'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:16.076513 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f47baebc-a61d-430f-aa39-e258b23f0e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147e44123a2b40caa56a9871ca92b3e9', 'request_id': 'req-f47baebc-a61d-430f-aa39-e258b23f0e49'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:16.091402 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f47baebc-a61d-430f-aa39-e258b23f0e49 admin admin] The token cache is being invalidate because project 147e44123a2b40caa56a9871ca92b3e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:42:16.091740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f47baebc-a61d-430f-aa39-e258b23f0e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:16.092422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f47baebc-a61d-430f-aa39-e258b23f0e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:16.094460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1151/2326] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:42:15 2022] DELETE /identity/v3/projects/147e44123a2b40caa56a9871ca92b3e9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:42:16.100022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117103e-1a7a-4124-9724-881e06bfbe76 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:16.100314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117103e-1a7a-4124-9724-881e06bfbe76 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:16.100595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117103e-1a7a-4124-9724-881e06bfbe76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:16.122259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e117103e-1a7a-4124-9724-881e06bfbe76 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:16.123475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e117103e-1a7a-4124-9724-881e06bfbe76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:16.142767 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:42:16.193757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1176/2327] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:42:16.202194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7fee-0b01-489e-bf64-f6a6b7d20559 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:16.202492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7fee-0b01-489e-bf64-f6a6b7d20559 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:16.202775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7fee-0b01-489e-bf64-f6a6b7d20559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:16.224473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-eafe7fee-0b01-489e-bf64-f6a6b7d20559 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:16.225609 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-eafe7fee-0b01-489e-bf64-f6a6b7d20559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:16.233706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1152/2328] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:42:16.239125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117103e-1a7a-4124-9724-881e06bfbe76 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:16.249524 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117103e-1a7a-4124-9724-881e06bfbe76 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:16.254182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08232356-f8b6-4495-8384-8d261eeaa5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:16.254835 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08232356-f8b6-4495-8384-8d261eeaa5ca admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:16.255119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08232356-f8b6-4495-8384-8d261eeaa5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:16.255398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08232356-f8b6-4495-8384-8d261eeaa5ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:16.256121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08232356-f8b6-4495-8384-8d261eeaa5ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:42:16.256432 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08232356-f8b6-4495-8384-8d261eeaa5ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:16.258054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08232356-f8b6-4495-8384-8d261eeaa5ca admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:16.269868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1177/2329] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:42:16.274310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe7fee-0b01-489e-bf64-f6a6b7d20559 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:16.284159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe7fee-0b01-489e-bf64-f6a6b7d20559 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:16.288859 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b1ea6-e958-4074-b613-ae524d209da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:16.289509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b1ea6-e958-4074-b613-ae524d209da3 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:16.289789 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b1ea6-e958-4074-b613-ae524d209da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:16.290094 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b1ea6-e958-4074-b613-ae524d209da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:16.290839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b1ea6-e958-4074-b613-ae524d209da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:16.292716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b1ea6-e958-4074-b613-ae524d209da3 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:16.322983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1153/2330] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:42:16 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:42:16.327151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08232356-f8b6-4495-8384-8d261eeaa5ca admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:16.337569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08232356-f8b6-4495-8384-8d261eeaa5ca admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:16.394904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da37e50-be95-4290-8a08-f9e32dbf96c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:16.395576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da37e50-be95-4290-8a08-f9e32dbf96c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:16.395861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da37e50-be95-4290-8a08-f9e32dbf96c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:16.396139 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da37e50-be95-4290-8a08-f9e32dbf96c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:16.396832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da37e50-be95-4290-8a08-f9e32dbf96c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:16.398806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da37e50-be95-4290-8a08-f9e32dbf96c2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:16.422543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1178/2331] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:42:16 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:42:16.427144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b1ea6-e958-4074-b613-ae524d209da3 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:16.437793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b1ea6-e958-4074-b613-ae524d209da3 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:16.442754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b440d2d-252f-44d2-99d9-307e873287f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:16.444390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b440d2d-252f-44d2-99d9-307e873287f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:16.444682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b440d2d-252f-44d2-99d9-307e873287f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:16.444960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b440d2d-252f-44d2-99d9-307e873287f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:16.445689 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b440d2d-252f-44d2-99d9-307e873287f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:16.447483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b440d2d-252f-44d2-99d9-307e873287f6 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:16.460966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1154/2332] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:42:16.462197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da37e50-be95-4290-8a08-f9e32dbf96c2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:16.473614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da37e50-be95-4290-8a08-f9e32dbf96c2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:16.478856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60f9f6-cc24-4196-be12-41ab18cd3279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:16.479471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60f9f6-cc24-4196-be12-41ab18cd3279 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:16.479761 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60f9f6-cc24-4196-be12-41ab18cd3279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:16.480038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60f9f6-cc24-4196-be12-41ab18cd3279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:16.480799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60f9f6-cc24-4196-be12-41ab18cd3279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:16.482429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60f9f6-cc24-4196-be12-41ab18cd3279 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:16.493544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1179/2333] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:42:16.497677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b440d2d-252f-44d2-99d9-307e873287f6 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:16.507774 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b440d2d-252f-44d2-99d9-307e873287f6 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:16.512548 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24255200-6fea-4de0-87f6-7e7d7e6acbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:16.513471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24255200-6fea-4de0-87f6-7e7d7e6acbf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:16.513782 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24255200-6fea-4de0-87f6-7e7d7e6acbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:16.514112 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24255200-6fea-4de0-87f6-7e7d7e6acbf5 admin admin] PATH_INFO: `/v3/projects/e335fce584ed4c2f89f3413538344d77/users/cc09afe05b024d0aa092fdfdc2e3cd73/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:16.529491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24255200-6fea-4de0-87f6-7e7d7e6acbf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e335fce584ed4c2f89f3413538344d77, user_id=cc09afe05b024d0aa092fdfdc2e3cd73, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:16.532190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24255200-6fea-4de0-87f6-7e7d7e6acbf5 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:16.546152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1155/2334] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:42:16 2022] PUT /identity/v3/projects/e335fce584ed4c2f89f3413538344d77/users/cc09afe05b024d0aa092fdfdc2e3cd73/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:42:16.557438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a74523-d1aa-4461-85e9-039b9a89b96a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:16.557735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a74523-d1aa-4461-85e9-039b9a89b96a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:16.558039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a74523-d1aa-4461-85e9-039b9a89b96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:16.589221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-29a74523-d1aa-4461-85e9-039b9a89b96a None None] MFA Rules not processed for user `cc09afe05b024d0aa092fdfdc2e3cd73`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:16.590555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-29a74523-d1aa-4461-85e9-039b9a89b96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:16.668138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1180/2335] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:42:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:42:16.680533 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24255200-6fea-4de0-87f6-7e7d7e6acbf5 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:16.682403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-24255200-6fea-4de0-87f6-7e7d7e6acbf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:16.707271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:42:16.751461 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24255200-6fea-4de0-87f6-7e7d7e6acbf5 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:16.755151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c725ef-ef55-4d10-89c9-e2aa7b309cfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:16.755825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c725ef-ef55-4d10-89c9-e2aa7b309cfe service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:16.756116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c725ef-ef55-4d10-89c9-e2aa7b309cfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:16.756399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c725ef-ef55-4d10-89c9-e2aa7b309cfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:16.765756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c725ef-ef55-4d10-89c9-e2aa7b309cfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:16.767800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c725ef-ef55-4d10-89c9-e2aa7b309cfe service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:16.778854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1156/2336] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:42:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:42:16.807965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a74523-d1aa-4461-85e9-039b9a89b96a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:16.832424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:42:16.886272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a74523-d1aa-4461-85e9-039b9a89b96a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:16.891359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8d756-82aa-41ee-afdf-459cc8ba2ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:16.893046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8d756-82aa-41ee-afdf-459cc8ba2ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:16.893376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8d756-82aa-41ee-afdf-459cc8ba2ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:16.893660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8d756-82aa-41ee-afdf-459cc8ba2ecd admin admin] PATH_INFO: `/v3/users/cc09afe05b024d0aa092fdfdc2e3cd73` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:16.895077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8d756-82aa-41ee-afdf-459cc8ba2ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc09afe05b024d0aa092fdfdc2e3cd73)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:16.897125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8d756-82aa-41ee-afdf-459cc8ba2ecd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:16.966828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-61d8d756-82aa-41ee-afdf-459cc8ba2ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc09afe05b024d0aa092fdfdc2e3cd73'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:16.986136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-61d8d756-82aa-41ee-afdf-459cc8ba2ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc09afe05b024d0aa092fdfdc2e3cd73'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:16.987089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-61d8d756-82aa-41ee-afdf-459cc8ba2ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc09afe05b024d0aa092fdfdc2e3cd73'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:16.995191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-61d8d756-82aa-41ee-afdf-459cc8ba2ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc09afe05b024d0aa092fdfdc2e3cd73'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:16.997139 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-61d8d756-82aa-41ee-afdf-459cc8ba2ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc09afe05b024d0aa092fdfdc2e3cd73'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:17.022433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1181/2337] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:42:16 2022] DELETE /identity/v3/users/cc09afe05b024d0aa092fdfdc2e3cd73 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:42:17.026893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225a0f8-8e17-4965-9d8e-676f5df4b463 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:17.027862 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225a0f8-8e17-4965-9d8e-676f5df4b463 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:17.028325 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225a0f8-8e17-4965-9d8e-676f5df4b463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:17.048832 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-7225a0f8-8e17-4965-9d8e-676f5df4b463 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:17.050490 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7225a0f8-8e17-4965-9d8e-676f5df4b463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:17.099732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf338b6-37d8-4845-9c46-33eca66b674b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:17.100021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf338b6-37d8-4845-9c46-33eca66b674b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:17.100298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf338b6-37d8-4845-9c46-33eca66b674b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:17.114273 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1157/2338] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:42:17.119429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-edf338b6-37d8-4845-9c46-33eca66b674b None None] MFA Rules not processed for user `e56531920fc548a3a37f63ca0d1e5d82`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:17.120631 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-edf338b6-37d8-4845-9c46-33eca66b674b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:17.124878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225a0f8-8e17-4965-9d8e-676f5df4b463 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:17.126796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7225a0f8-8e17-4965-9d8e-676f5df4b463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:17.178770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1182/2339] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:42:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:42:17.185679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225a0f8-8e17-4965-9d8e-676f5df4b463 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:17.190575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdc4c4-1b60-4abb-b493-1848dc63dd8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:17.191510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdc4c4-1b60-4abb-b493-1848dc63dd8f service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:17.191510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdc4c4-1b60-4abb-b493-1848dc63dd8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:17.191690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdc4c4-1b60-4abb-b493-1848dc63dd8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:17.201443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdc4c4-1b60-4abb-b493-1848dc63dd8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:17.202956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdc4c4-1b60-4abb-b493-1848dc63dd8f service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:17.213036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1158/2340] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:42:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:42:17.601669 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf338b6-37d8-4845-9c46-33eca66b674b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:17.603121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-edf338b6-37d8-4845-9c46-33eca66b674b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:17.618385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf338b6-37d8-4845-9c46-33eca66b674b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:17.623091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80076e-784c-4e22-a252-a5a5a3bcc561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:17.623897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80076e-784c-4e22-a252-a5a5a3bcc561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:17.624177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80076e-784c-4e22-a252-a5a5a3bcc561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:17.624451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80076e-784c-4e22-a252-a5a5a3bcc561 admin admin] PATH_INFO: `/v3/projects/e335fce584ed4c2f89f3413538344d77` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:17.625762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80076e-784c-4e22-a252-a5a5a3bcc561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e335fce584ed4c2f89f3413538344d77)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:17.627971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80076e-784c-4e22-a252-a5a5a3bcc561 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:17.671198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:42:17.682172 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-af80076e-784c-4e22-a252-a5a5a3bcc561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e335fce584ed4c2f89f3413538344d77', 'request_id': 'req-af80076e-784c-4e22-a252-a5a5a3bcc561'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:17.690071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-af80076e-784c-4e22-a252-a5a5a3bcc561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e335fce584ed4c2f89f3413538344d77', 'request_id': 'req-af80076e-784c-4e22-a252-a5a5a3bcc561'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:17.696323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-af80076e-784c-4e22-a252-a5a5a3bcc561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e335fce584ed4c2f89f3413538344d77', 'request_id': 'req-af80076e-784c-4e22-a252-a5a5a3bcc561'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:17.707062 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-af80076e-784c-4e22-a252-a5a5a3bcc561 admin admin] The token cache is being invalidate because project e335fce584ed4c2f89f3413538344d77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:42:17.707347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-af80076e-784c-4e22-a252-a5a5a3bcc561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:17.707997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-af80076e-784c-4e22-a252-a5a5a3bcc561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:42:17.709539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1183/2341] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:42:17 2022] DELETE /identity/v3/projects/e335fce584ed4c2f89f3413538344d77 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:42:17.714691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c0f6d-96f0-44bc-a5ed-f93db473dbb6 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:17.715168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c0f6d-96f0-44bc-a5ed-f93db473dbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:17.715266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c0f6d-96f0-44bc-a5ed-f93db473dbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:17.734560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-210c0f6d-96f0-44bc-a5ed-f93db473dbb6 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:17.737248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-210c0f6d-96f0-44bc-a5ed-f93db473dbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:17.799222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1159/2342] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:42:17.808110 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3eddc-ae2e-43f1-a012-f0bb114f3ecf None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:17.808399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3eddc-ae2e-43f1-a012-f0bb114f3ecf None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:17.808670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3eddc-ae2e-43f1-a012-f0bb114f3ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:17.828680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-79d3eddc-ae2e-43f1-a012-f0bb114f3ecf None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:17.830459 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-79d3eddc-ae2e-43f1-a012-f0bb114f3ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:17.839280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1184/2343] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:42:17.843313 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c0f6d-96f0-44bc-a5ed-f93db473dbb6 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:17.853456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c0f6d-96f0-44bc-a5ed-f93db473dbb6 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:17.859143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499fd1e-1a81-4ff3-8080-67c4af621249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:17.860095 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499fd1e-1a81-4ff3-8080-67c4af621249 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:17.860530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499fd1e-1a81-4ff3-8080-67c4af621249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:17.860953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499fd1e-1a81-4ff3-8080-67c4af621249 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:17.861704 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499fd1e-1a81-4ff3-8080-67c4af621249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:42:17.862168 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499fd1e-1a81-4ff3-8080-67c4af621249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:17.863785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499fd1e-1a81-4ff3-8080-67c4af621249 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:17.866466 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:42:17.883086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1160/2344] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:42:17.887326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3eddc-ae2e-43f1-a012-f0bb114f3ecf None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:17.897294 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3eddc-ae2e-43f1-a012-f0bb114f3ecf None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:17.902805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96666ae5-f0cb-43a4-a096-9d9358fb98b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:17.903472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96666ae5-f0cb-43a4-a096-9d9358fb98b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:17.903775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96666ae5-f0cb-43a4-a096-9d9358fb98b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:17.904009 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96666ae5-f0cb-43a4-a096-9d9358fb98b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:17.904841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96666ae5-f0cb-43a4-a096-9d9358fb98b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:17.906826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96666ae5-f0cb-43a4-a096-9d9358fb98b8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:17.945856 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1185/2345] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:42:17 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:42:17.947076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499fd1e-1a81-4ff3-8080-67c4af621249 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:17.958897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499fd1e-1a81-4ff3-8080-67c4af621249 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:18.044705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0211171-23ea-46df-94df-e10ba6cfe78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:18.045553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0211171-23ea-46df-94df-e10ba6cfe78f admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:18.045971 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0211171-23ea-46df-94df-e10ba6cfe78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:18.046279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0211171-23ea-46df-94df-e10ba6cfe78f admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:18.047001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0211171-23ea-46df-94df-e10ba6cfe78f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:18.049456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0211171-23ea-46df-94df-e10ba6cfe78f admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:18.074494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1161/2346] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:42:17 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:42:18.075555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96666ae5-f0cb-43a4-a096-9d9358fb98b8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:18.087263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96666ae5-f0cb-43a4-a096-9d9358fb98b8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:18.091997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80866117-bb5d-4d82-b865-d81cd40cdb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:18.092662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80866117-bb5d-4d82-b865-d81cd40cdb3f admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:18.092946 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80866117-bb5d-4d82-b865-d81cd40cdb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:18.093223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80866117-bb5d-4d82-b865-d81cd40cdb3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:18.094145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80866117-bb5d-4d82-b865-d81cd40cdb3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:18.096351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80866117-bb5d-4d82-b865-d81cd40cdb3f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:18.107046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1186/2347] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:42:18.111285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0211171-23ea-46df-94df-e10ba6cfe78f admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:18.121335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0211171-23ea-46df-94df-e10ba6cfe78f admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:18.125984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e4e17-0d88-4824-87ca-a9241cd5ebfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:18.126757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e4e17-0d88-4824-87ca-a9241cd5ebfe admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:18.127029 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e4e17-0d88-4824-87ca-a9241cd5ebfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:18.127307 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e4e17-0d88-4824-87ca-a9241cd5ebfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:18.128037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e4e17-0d88-4824-87ca-a9241cd5ebfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:18.129664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e4e17-0d88-4824-87ca-a9241cd5ebfe admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:18.146083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1162/2348] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:42:18.147201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80866117-bb5d-4d82-b865-d81cd40cdb3f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:18.160274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80866117-bb5d-4d82-b865-d81cd40cdb3f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:18.164895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7b3b5-ee29-4acd-8dad-21817856327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:18.165767 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7b3b5-ee29-4acd-8dad-21817856327e admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:18.166081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7b3b5-ee29-4acd-8dad-21817856327e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:18.166376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7b3b5-ee29-4acd-8dad-21817856327e admin admin] PATH_INFO: `/v3/projects/8260111bf70a4afab166caf5a4881931/users/f5032f6d4f074cae838066344fcea598/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:18.188471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7b3b5-ee29-4acd-8dad-21817856327e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8260111bf70a4afab166caf5a4881931, user_id=f5032f6d4f074cae838066344fcea598, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:18.191898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7b3b5-ee29-4acd-8dad-21817856327e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:18.206399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1187/2349] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:42:18 2022] PUT /identity/v3/projects/8260111bf70a4afab166caf5a4881931/users/f5032f6d4f074cae838066344fcea598/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:42:18.211333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa984173-48e3-49d8-ad40-4f909192303e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:18.211619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa984173-48e3-49d8-ad40-4f909192303e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:18.211931 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa984173-48e3-49d8-ad40-4f909192303e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:18.231310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-aa984173-48e3-49d8-ad40-4f909192303e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:18.232798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-aa984173-48e3-49d8-ad40-4f909192303e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:18.297172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1163/2350] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:42:18.308717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7b3b5-ee29-4acd-8dad-21817856327e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:18.310442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f5c7b3b5-ee29-4acd-8dad-21817856327e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:18.366909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7b3b5-ee29-4acd-8dad-21817856327e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:18.370472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2ed0b-cf9d-4324-bed4-d0c5368a9bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:18.371114 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2ed0b-cf9d-4324-bed4-d0c5368a9bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:18.371396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2ed0b-cf9d-4324-bed4-d0c5368a9bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:18.371674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2ed0b-cf9d-4324-bed4-d0c5368a9bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:18.384101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2ed0b-cf9d-4324-bed4-d0c5368a9bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:18.385814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2ed0b-cf9d-4324-bed4-d0c5368a9bb4 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:18.396290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1188/2351] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:42:22.334397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6c335-f248-4388-9cb4-630cfc6c49d9 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:22.334920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6c335-f248-4388-9cb4-630cfc6c49d9 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:22.335018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6c335-f248-4388-9cb4-630cfc6c49d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:22.362867 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-1ce6c335-f248-4388-9cb4-630cfc6c49d9 None None] MFA Rules not processed for user `f5032f6d4f074cae838066344fcea598`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:22.364012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-1ce6c335-f248-4388-9cb4-630cfc6c49d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:22.443437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1164/2352] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:42:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:42:22.447687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2ed0b-cf9d-4324-bed4-d0c5368a9bb4 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:22.459917 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2ed0b-cf9d-4324-bed4-d0c5368a9bb4 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:22.475469 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78191a-1b3b-48da-b303-420d6d8b99a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:22.476248 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78191a-1b3b-48da-b303-420d6d8b99a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:22.476655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78191a-1b3b-48da-b303-420d6d8b99a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:22.476998 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78191a-1b3b-48da-b303-420d6d8b99a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:22.477764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78191a-1b3b-48da-b303-420d6d8b99a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:22.479786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78191a-1b3b-48da-b303-420d6d8b99a7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:22.517984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1189/2353] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:42:22 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:42:22.519504 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6c335-f248-4388-9cb4-630cfc6c49d9 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:22.525564 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:42:22.533682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6c335-f248-4388-9cb4-630cfc6c49d9 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:22.604203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31bada-533d-4adb-adb9-df03b43fe215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:22.604880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31bada-533d-4adb-adb9-df03b43fe215 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:22.605167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31bada-533d-4adb-adb9-df03b43fe215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:22.605448 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31bada-533d-4adb-adb9-df03b43fe215 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:22.606209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31bada-533d-4adb-adb9-df03b43fe215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:22.608146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31bada-533d-4adb-adb9-df03b43fe215 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:22.636731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1165/2354] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:42:22 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:42:22.640816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78191a-1b3b-48da-b303-420d6d8b99a7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:22.651827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78191a-1b3b-48da-b303-420d6d8b99a7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:22.656893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76407bd9-9e37-4693-b925-94313a0dfe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:22.657602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76407bd9-9e37-4693-b925-94313a0dfe17 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:22.657847 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76407bd9-9e37-4693-b925-94313a0dfe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:22.658170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76407bd9-9e37-4693-b925-94313a0dfe17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:22.659287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76407bd9-9e37-4693-b925-94313a0dfe17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:22.660558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76407bd9-9e37-4693-b925-94313a0dfe17 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:22.683031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1190/2355] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:42:22.687468 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31bada-533d-4adb-adb9-df03b43fe215 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:22.703852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31bada-533d-4adb-adb9-df03b43fe215 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:22.708814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819f8b4-9771-414c-a135-82e34c3e2766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:22.709732 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819f8b4-9771-414c-a135-82e34c3e2766 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:22.710062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819f8b4-9771-414c-a135-82e34c3e2766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:22.710344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819f8b4-9771-414c-a135-82e34c3e2766 admin admin] PATH_INFO: `/v3/projects/84f6a81ab18042898f1762057c006de3/users/8fdc221326054df385a450b89afbe8ac/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:22.725165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819f8b4-9771-414c-a135-82e34c3e2766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f6a81ab18042898f1762057c006de3, user_id=8fdc221326054df385a450b89afbe8ac, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:22.727870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819f8b4-9771-414c-a135-82e34c3e2766 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:22.746967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1166/2356] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:42:22 2022] PUT /identity/v3/projects/84f6a81ab18042898f1762057c006de3/users/8fdc221326054df385a450b89afbe8ac/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:42:22.750685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76407bd9-9e37-4693-b925-94313a0dfe17 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:22.762716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76407bd9-9e37-4693-b925-94313a0dfe17 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:22.793332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:42:22.827494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2cf53-9d9f-491c-9ac7-59fc73dc2573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:22.828190 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2cf53-9d9f-491c-9ac7-59fc73dc2573 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:22.828449 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2cf53-9d9f-491c-9ac7-59fc73dc2573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:22.828720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2cf53-9d9f-491c-9ac7-59fc73dc2573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:22.830054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2cf53-9d9f-491c-9ac7-59fc73dc2573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:22.831663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2cf53-9d9f-491c-9ac7-59fc73dc2573 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:22.841938 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1191/2357] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:42:22.846161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819f8b4-9771-414c-a135-82e34c3e2766 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:22.856680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819f8b4-9771-414c-a135-82e34c3e2766 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:22.861737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445e939-7288-4220-87d8-4eb17162abe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:22.862659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445e939-7288-4220-87d8-4eb17162abe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:22.862936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445e939-7288-4220-87d8-4eb17162abe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:22.863209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445e939-7288-4220-87d8-4eb17162abe0 admin admin] PATH_INFO: `/v3/projects/84f6a81ab18042898f1762057c006de3/users/8fdc221326054df385a450b89afbe8ac/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:22.865176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445e939-7288-4220-87d8-4eb17162abe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f6a81ab18042898f1762057c006de3, user_id=8fdc221326054df385a450b89afbe8ac, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:22.867852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445e939-7288-4220-87d8-4eb17162abe0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:22.879543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1167/2358] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:42:22 2022] PUT /identity/v3/projects/84f6a81ab18042898f1762057c006de3/users/8fdc221326054df385a450b89afbe8ac/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:42:28.528669 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05b890-a276-4fd1-ae02-7958579a78f0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:28.529219 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05b890-a276-4fd1-ae02-7958579a78f0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:28.529537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05b890-a276-4fd1-ae02-7958579a78f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:28.577257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-8f05b890-a276-4fd1-ae02-7958579a78f0 None None] MFA Rules not processed for user `8fdc221326054df385a450b89afbe8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:28.579093 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8f05b890-a276-4fd1-ae02-7958579a78f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:28.680904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1192/2359] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:42:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:42:28.699207 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445e939-7288-4220-87d8-4eb17162abe0 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:28.700982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e445e939-7288-4220-87d8-4eb17162abe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:28.814608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445e939-7288-4220-87d8-4eb17162abe0 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:28.818530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c82a6-d6cb-40eb-8135-a1497267c4ba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b07dcd8c0de4d1694d4363520726df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:28.819181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c82a6-d6cb-40eb-8135-a1497267c4ba service glance] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:28.819464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c82a6-d6cb-40eb-8135-a1497267c4ba service glance] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:28.819737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c82a6-d6cb-40eb-8135-a1497267c4ba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:28.895720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c82a6-d6cb-40eb-8135-a1497267c4ba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:28.897385 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c82a6-d6cb-40eb-8135-a1497267c4ba service glance] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:28.930457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1168/2360] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:42:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:42:29.069587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87d545-a486-46b6-a7ab-8d12002070e8 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:29.069878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87d545-a486-46b6-a7ab-8d12002070e8 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:29.070203 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87d545-a486-46b6-a7ab-8d12002070e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:29.089743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-6b87d545-a486-46b6-a7ab-8d12002070e8 None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:29.090992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6b87d545-a486-46b6-a7ab-8d12002070e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:29.165750 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1193/2361] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:42:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:42:29.217035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c82a6-d6cb-40eb-8135-a1497267c4ba service glance] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:29.218962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9f5c82a6-d6cb-40eb-8135-a1497267c4ba service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:29.231319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:42:29.336589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c82a6-d6cb-40eb-8135-a1497267c4ba service glance] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:29.343492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab4c78-1a85-4b3e-b664-928f2026a895 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:29.344045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab4c78-1a85-4b3e-b664-928f2026a895 service swift] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:29.344387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab4c78-1a85-4b3e-b664-928f2026a895 service swift] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:29.345650 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab4c78-1a85-4b3e-b664-928f2026a895 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:29.356871 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab4c78-1a85-4b3e-b664-928f2026a895 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:29.357281 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab4c78-1a85-4b3e-b664-928f2026a895 service swift] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:29.381833 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1169/2362] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:42:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 12:42:38.866113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177e078-d149-45a5-b04e-b98a4cddad40 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:38.866725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177e078-d149-45a5-b04e-b98a4cddad40 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:38.866725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177e078-d149-45a5-b04e-b98a4cddad40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:38.886306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2177e078-d149-45a5-b04e-b98a4cddad40 None None] MFA Rules not processed for user `f5032f6d4f074cae838066344fcea598`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:38.887506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2177e078-d149-45a5-b04e-b98a4cddad40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:38.894235 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1194/2363] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:42:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:42:39.370223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=28 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:42:48.881267 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=77 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:42:53.455422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab4c78-1a85-4b3e-b664-928f2026a895 service swift] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:53.457993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a6ab4c78-1a85-4b3e-b664-928f2026a895 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:53.531714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab4c78-1a85-4b3e-b664-928f2026a895 service swift] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:53.537788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c649e-77e8-4017-84c4-2dd103c2f885 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:53.538764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c649e-77e8-4017-84c4-2dd103c2f885 service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:53.539062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c649e-77e8-4017-84c4-2dd103c2f885 service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:53.539734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c649e-77e8-4017-84c4-2dd103c2f885 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:53.541476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0b0c649e-77e8-4017-84c4-2dd103c2f885 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:53.555071 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c649e-77e8-4017-84c4-2dd103c2f885 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:53.556701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c649e-77e8-4017-84c4-2dd103c2f885 service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:53.567307 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1170/2364] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:42:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:42:55.443827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacb0b4-25f4-4a96-91b6-8aa89c6f5904 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:55.444318 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacb0b4-25f4-4a96-91b6-8aa89c6f5904 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:55.444498 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacb0b4-25f4-4a96-91b6-8aa89c6f5904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:55.464155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2eacb0b4-25f4-4a96-91b6-8aa89c6f5904 None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:42:55.465361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2eacb0b4-25f4-4a96-91b6-8aa89c6f5904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:42:55.472819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1195/2365] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:42:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:42:55.485309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c649e-77e8-4017-84c4-2dd103c2f885 service placement] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:42:55.496944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c649e-77e8-4017-84c4-2dd103c2f885 service placement] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:42:55.504575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b54dc9-e336-434c-b3eb-6a65d1f075b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:42:55.505330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b54dc9-e336-434c-b3eb-6a65d1f075b4 service swift] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:42:55.505927 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b54dc9-e336-434c-b3eb-6a65d1f075b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:42:55.506188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b54dc9-e336-434c-b3eb-6a65d1f075b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:42:55.520441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b54dc9-e336-434c-b3eb-6a65d1f075b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:42:55.520441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b54dc9-e336-434c-b3eb-6a65d1f075b4 service swift] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:42:55.531268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1171/2366] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:42:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:43:00.687243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eacb0b4-25f4-4a96-91b6-8aa89c6f5904 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:43:00.688692 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2eacb0b4-25f4-4a96-91b6-8aa89c6f5904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:43:00.764919 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eacb0b4-25f4-4a96-91b6-8aa89c6f5904 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:43:00.770271 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:43:00.771263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:43:00.771535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:43:00.775668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] PATH_INFO: `/v3/users/319718e05ce84d16953162fcaacf1d8f` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:43:00.775668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=319718e05ce84d16953162fcaacf1d8f)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:43:00.775668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:43:00.825480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '319718e05ce84d16953162fcaacf1d8f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:43:00.832287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '319718e05ce84d16953162fcaacf1d8f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:43:00.833136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '319718e05ce84d16953162fcaacf1d8f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:43:00.840399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '319718e05ce84d16953162fcaacf1d8f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:43:00.841060 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '319718e05ce84d16953162fcaacf1d8f'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:43:00.865228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1196/2367] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:43:00 2022] DELETE /identity/v3/users/319718e05ce84d16953162fcaacf1d8f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:43:00.869690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ed8fa-826c-4da2-b721-2fa0222f39bd None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:43:00.870253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ed8fa-826c-4da2-b721-2fa0222f39bd None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:43:00.870358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ed8fa-826c-4da2-b721-2fa0222f39bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:43:00.889794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-288ed8fa-826c-4da2-b721-2fa0222f39bd None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:43:00.891045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-288ed8fa-826c-4da2-b721-2fa0222f39bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:43:00.964751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1172/2368] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:43:00.975671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:43:00.977149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:43:01.054784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3f523-a8e9-41c0-9b17-b238718868b2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:43:01.059519 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a8dc9-634e-4a05-a0c2-8015365e475b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:43:01.060265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a8dc9-634e-4a05-a0c2-8015365e475b service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:43:01.060610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a8dc9-634e-4a05-a0c2-8015365e475b service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:43:01.061124 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a8dc9-634e-4a05-a0c2-8015365e475b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:43:01.071330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a8dc9-634e-4a05-a0c2-8015365e475b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:43:01.072824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a8dc9-634e-4a05-a0c2-8015365e475b service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:43:01.083504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1197/2369] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:43:01.396707 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ed8fa-826c-4da2-b721-2fa0222f39bd None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:43:01.399041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-288ed8fa-826c-4da2-b721-2fa0222f39bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:43:01.413783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ed8fa-826c-4da2-b721-2fa0222f39bd None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:43:01.421265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:43:01.422693 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:43:01.423061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:43:01.423516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] PATH_INFO: `/v3/projects/e97970687da94cd8a4aef929b07bf2ff` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:43:01.424961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e97970687da94cd8a4aef929b07bf2ff)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:43:01.426920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:43:01.495137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e97970687da94cd8a4aef929b07bf2ff', 'request_id': 'req-6fad4b83-45bd-4947-8ed7-44ad09cd679c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:43:01.502922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e97970687da94cd8a4aef929b07bf2ff', 'request_id': 'req-6fad4b83-45bd-4947-8ed7-44ad09cd679c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:43:01.512907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e97970687da94cd8a4aef929b07bf2ff', 'request_id': 'req-6fad4b83-45bd-4947-8ed7-44ad09cd679c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:43:01.531035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:43:01.535671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] The token cache is being invalidate because project e97970687da94cd8a4aef929b07bf2ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:43:01.536200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:43:01.537012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:43:01.538902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1173/2370] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:43:01 2022] DELETE /identity/v3/projects/e97970687da94cd8a4aef929b07bf2ff => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:43:11.078301 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=82,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:43:34.143014 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dc815-a3f9-461f-969f-438734b2af00 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:43:34.143014 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dc815-a3f9-461f-969f-438734b2af00 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:43:34.143014 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dc815-a3f9-461f-969f-438734b2af00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:43:34.152465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7b8dc815-a3f9-461f-969f-438734b2af00 None None] MFA Rules not processed for user `0f4376954dbd4402ae5a990f1a56baf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:43:34.153660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7b8dc815-a3f9-461f-969f-438734b2af00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:43:34.208261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1198/2371] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:43:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:43:44.201056 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=25 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:43:45.058338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:43:45.060161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:43:45.122042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad4b83-45bd-4947-8ed7-44ad09cd679c admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:43:45.125597 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8acb0-42bc-4e84-9325-bcbf49fbbe7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:43:45.126285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8acb0-42bc-4e84-9325-bcbf49fbbe7e service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:43:45.126907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8acb0-42bc-4e84-9325-bcbf49fbbe7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:43:45.126907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8acb0-42bc-4e84-9325-bcbf49fbbe7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:43:45.128600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-57f8acb0-42bc-4e84-9325-bcbf49fbbe7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:43:45.205825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8acb0-42bc-4e84-9325-bcbf49fbbe7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:43:45.207342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8acb0-42bc-4e84-9325-bcbf49fbbe7e service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:43:45.229973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1174/2372] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:43:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:43:55.213555 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=50 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:44:15.172943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc898f-d5dc-491d-a020-cb0e7283141b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:15.173863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc898f-d5dc-491d-a020-cb0e7283141b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:15.174171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc898f-d5dc-491d-a020-cb0e7283141b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:15.196180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e3dc898f-d5dc-491d-a020-cb0e7283141b None None] MFA Rules not processed for user `6f611c60491941ca8a0200212ab0c7a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:44:15.197567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e3dc898f-d5dc-491d-a020-cb0e7283141b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:15.257520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1199/2373] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:44:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:44:17.136483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd55fc-90bc-4d3d-a104-5f10eeb2e2ec None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:17.137020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd55fc-90bc-4d3d-a104-5f10eeb2e2ec None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:17.137132 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd55fc-90bc-4d3d-a104-5f10eeb2e2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:17.159630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-2edd55fc-90bc-4d3d-a104-5f10eeb2e2ec None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:44:17.160918 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2edd55fc-90bc-4d3d-a104-5f10eeb2e2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:17.240657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1175/2374] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:44:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:44:17.296197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc898f-d5dc-491d-a020-cb0e7283141b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:44:17.298206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e3dc898f-d5dc-491d-a020-cb0e7283141b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:17.379133 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc898f-d5dc-491d-a020-cb0e7283141b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:44:17.384836 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c01de-807c-46de-a41c-7f0eb1c918f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:44:17.385484 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c01de-807c-46de-a41c-7f0eb1c918f6 service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:17.385765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c01de-807c-46de-a41c-7f0eb1c918f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:17.386086 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c01de-807c-46de-a41c-7f0eb1c918f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:17.394374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c01de-807c-46de-a41c-7f0eb1c918f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:44:17.396548 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c01de-807c-46de-a41c-7f0eb1c918f6 service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:44:17.408533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1200/2375] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:44:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:44:27.229416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=25 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:44:27.401384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=57 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:44:30.592271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd55fc-90bc-4d3d-a104-5f10eeb2e2ec None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:44:30.593756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2edd55fc-90bc-4d3d-a104-5f10eeb2e2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:30.605354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd55fc-90bc-4d3d-a104-5f10eeb2e2ec None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:44:30.609222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e103dab-c856-4b01-bb58-643eeda509e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:44:30.609885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e103dab-c856-4b01-bb58-643eeda509e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:30.610190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e103dab-c856-4b01-bb58-643eeda509e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:30.610464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e103dab-c856-4b01-bb58-643eeda509e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:30.612373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5e103dab-c856-4b01-bb58-643eeda509e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:30.682631 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e103dab-c856-4b01-bb58-643eeda509e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:44:30.684004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e103dab-c856-4b01-bb58-643eeda509e0 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:44:30.695383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1176/2376] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:44:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:44:34.952346 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c01de-807c-46de-a41c-7f0eb1c918f6 service swift] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:44:34.953786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-866c01de-807c-46de-a41c-7f0eb1c918f6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:35.016623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c01de-807c-46de-a41c-7f0eb1c918f6 service swift] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:44:35.021437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0366e-0ba1-489f-9c45-7f6d6278c054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:44:35.022121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0366e-0ba1-489f-9c45-7f6d6278c054 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:35.022404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0366e-0ba1-489f-9c45-7f6d6278c054 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:35.022679 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0366e-0ba1-489f-9c45-7f6d6278c054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:35.024512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ced0366e-0ba1-489f-9c45-7f6d6278c054 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:35.040607 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0366e-0ba1-489f-9c45-7f6d6278c054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:44:35.042789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0366e-0ba1-489f-9c45-7f6d6278c054 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:44:35.056006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1201/2377] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:44:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:44:35.633725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bbd96-de5e-42bb-b070-76d08b495fb4 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:35.634290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bbd96-de5e-42bb-b070-76d08b495fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:35.634290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bbd96-de5e-42bb-b070-76d08b495fb4 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:35.635846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1177/2378] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:44:35.642276 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5183d1a-d320-47f1-91be-63b4ebbc2191 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:35.642567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5183d1a-d320-47f1-91be-63b4ebbc2191 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:35.642843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5183d1a-d320-47f1-91be-63b4ebbc2191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:35.664912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c5183d1a-d320-47f1-91be-63b4ebbc2191 None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:44:35.666456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c5183d1a-d320-47f1-91be-63b4ebbc2191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:35.677965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1202/2379] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:44:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:44:35.694156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bbd96-de5e-42bb-b070-76d08b495fb4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:44:35.696464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0c0bbd96-de5e-42bb-b070-76d08b495fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:35.766786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bbd96-de5e-42bb-b070-76d08b495fb4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:44:35.770379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa53cb2-36ad-442b-989b-14a3c60de08e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b07dcd8c0de4d1694d4363520726df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:44:35.771040 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa53cb2-36ad-442b-989b-14a3c60de08e service glance] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:35.771325 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa53cb2-36ad-442b-989b-14a3c60de08e service glance] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:35.771598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa53cb2-36ad-442b-989b-14a3c60de08e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:35.783375 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa53cb2-36ad-442b-989b-14a3c60de08e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:44:35.784831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa53cb2-36ad-442b-989b-14a3c60de08e service glance] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:44:35.796328 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1178/2380] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:44:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:44:35.870134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5321b-1db2-48c3-b225-b015b6a1e4b2 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:35.870424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5321b-1db2-48c3-b225-b015b6a1e4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:35.870709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5321b-1db2-48c3-b225-b015b6a1e4b2 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:35.872097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1203/2381] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:44:35.877169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f0925-a5a1-4202-9cc7-1e538a97ecb7 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:35.877454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f0925-a5a1-4202-9cc7-1e538a97ecb7 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:35.877732 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f0925-a5a1-4202-9cc7-1e538a97ecb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:35.898770 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-745f0925-a5a1-4202-9cc7-1e538a97ecb7 None None] MFA Rules not processed for user `375d321d6de2494794254e50151c02c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:44:35.899948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-745f0925-a5a1-4202-9cc7-1e538a97ecb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:35.906870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1179/2382] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:44:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:44:35.919402 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5321b-1db2-48c3-b225-b015b6a1e4b2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:44:35.931692 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5321b-1db2-48c3-b225-b015b6a1e4b2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:44:35.935274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865643c3-b0a7-4447-a4e1-864588171a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:44:35.935921 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865643c3-b0a7-4447-a4e1-864588171a09 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:35.936352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865643c3-b0a7-4447-a4e1-864588171a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:35.936633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865643c3-b0a7-4447-a4e1-864588171a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:35.944653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865643c3-b0a7-4447-a4e1-864588171a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:44:35.946127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865643c3-b0a7-4447-a4e1-864588171a09 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:44:35.962861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1204/2383] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:44:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:44:36.050961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b418bde-48ea-4366-8fe5-7c007a7b572c None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:36.051371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b418bde-48ea-4366-8fe5-7c007a7b572c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:36.051905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b418bde-48ea-4366-8fe5-7c007a7b572c None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:36.054044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1180/2384] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:44:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:44:36.062420 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114ba57-75a6-4440-bdbc-7097760d12d9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:36.062756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114ba57-75a6-4440-bdbc-7097760d12d9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:36.063036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114ba57-75a6-4440-bdbc-7097760d12d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:36.086533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c114ba57-75a6-4440-bdbc-7097760d12d9 None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:44:36.087751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c114ba57-75a6-4440-bdbc-7097760d12d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:36.097270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1205/2385] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:44:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:44:36.109163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b418bde-48ea-4366-8fe5-7c007a7b572c None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:44:36.122070 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b418bde-48ea-4366-8fe5-7c007a7b572c None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:44:36.126263 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa37a7-8871-4b19-a1fb-8a661ecd78db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:44:36.126910 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa37a7-8871-4b19-a1fb-8a661ecd78db service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:36.127194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa37a7-8871-4b19-a1fb-8a661ecd78db service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:36.127471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa37a7-8871-4b19-a1fb-8a661ecd78db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:36.138822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa37a7-8871-4b19-a1fb-8a661ecd78db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:44:36.140267 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa37a7-8871-4b19-a1fb-8a661ecd78db service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:44:36.151934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1181/2386] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:44:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:44:37.965346 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114ba57-75a6-4440-bdbc-7097760d12d9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:44:37.966802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c114ba57-75a6-4440-bdbc-7097760d12d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:37.980739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114ba57-75a6-4440-bdbc-7097760d12d9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:44:37.986855 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d733d36-d35c-413e-a8b8-d67cbe812cc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:44:37.987560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d733d36-d35c-413e-a8b8-d67cbe812cc6 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:37.987792 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d733d36-d35c-413e-a8b8-d67cbe812cc6 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:37.988066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d733d36-d35c-413e-a8b8-d67cbe812cc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:37.990468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0d733d36-d35c-413e-a8b8-d67cbe812cc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:38.056006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d733d36-d35c-413e-a8b8-d67cbe812cc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:44:38.057667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d733d36-d35c-413e-a8b8-d67cbe812cc6 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:44:38.067660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1206/2387] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:44:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:44:46.137415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e17a-4a68-4ee5-8c6a-91fbbee83795 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:46.138012 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e17a-4a68-4ee5-8c6a-91fbbee83795 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:46.138125 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e17a-4a68-4ee5-8c6a-91fbbee83795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:46.159944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-7004e17a-4a68-4ee5-8c6a-91fbbee83795 None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:44:46.161261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7004e17a-4a68-4ee5-8c6a-91fbbee83795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:46.169249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1182/2388] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:44:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:44:46.183297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d733d36-d35c-413e-a8b8-d67cbe812cc6 service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:44:46.195975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d733d36-d35c-413e-a8b8-d67cbe812cc6 service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:44:46.201055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd506c02-3843-4b1e-82d7-ee25e8ebf01b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:44:46.201724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd506c02-3843-4b1e-82d7-ee25e8ebf01b service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:46.202054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd506c02-3843-4b1e-82d7-ee25e8ebf01b service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:46.202366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd506c02-3843-4b1e-82d7-ee25e8ebf01b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:46.211210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd506c02-3843-4b1e-82d7-ee25e8ebf01b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:44:46.212805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd506c02-3843-4b1e-82d7-ee25e8ebf01b service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:44:46.221359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1207/2389] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:44:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:44:54.334894 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004e17a-4a68-4ee5-8c6a-91fbbee83795 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:44:54.336614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7004e17a-4a68-4ee5-8c6a-91fbbee83795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:44:54.430369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004e17a-4a68-4ee5-8c6a-91fbbee83795 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:44:54.435605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:44:54.436633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:54.436942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:54.437264 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] PATH_INFO: `/v3/users/0f4376954dbd4402ae5a990f1a56baf3` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:54.438948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f4376954dbd4402ae5a990f1a56baf3)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:44:54.440933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:44:54.443947 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:44:54.501637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f4376954dbd4402ae5a990f1a56baf3'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:44:54.502464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f4376954dbd4402ae5a990f1a56baf3'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:44:54.503184 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f4376954dbd4402ae5a990f1a56baf3'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:44:54.513890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f4376954dbd4402ae5a990f1a56baf3'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:44:54.526903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f4376954dbd4402ae5a990f1a56baf3'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:44:54.553364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1183/2390] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:44:54 2022] DELETE /identity/v3/users/0f4376954dbd4402ae5a990f1a56baf3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:44:54.773998 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd506c02-3843-4b1e-82d7-ee25e8ebf01b service swift] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:44:54.789629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd506c02-3843-4b1e-82d7-ee25e8ebf01b service swift] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:44:54.879013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70038f6-e105-478a-9719-17f48da26d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:44:54.879806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70038f6-e105-478a-9719-17f48da26d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:44:54.880090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70038f6-e105-478a-9719-17f48da26d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:44:54.880367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70038f6-e105-478a-9719-17f48da26d4a admin admin] PATH_INFO: `/v3/projects/5d2fa8874dab4d3bbc87161b540ffae9` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:44:54.882057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70038f6-e105-478a-9719-17f48da26d4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d2fa8874dab4d3bbc87161b540ffae9)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:44:54.884330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70038f6-e105-478a-9719-17f48da26d4a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:44:54.887078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:44:54.941683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e70038f6-e105-478a-9719-17f48da26d4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d2fa8874dab4d3bbc87161b540ffae9', 'request_id': 'req-e70038f6-e105-478a-9719-17f48da26d4a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:44:54.949165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e70038f6-e105-478a-9719-17f48da26d4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d2fa8874dab4d3bbc87161b540ffae9', 'request_id': 'req-e70038f6-e105-478a-9719-17f48da26d4a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:44:54.955578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e70038f6-e105-478a-9719-17f48da26d4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d2fa8874dab4d3bbc87161b540ffae9', 'request_id': 'req-e70038f6-e105-478a-9719-17f48da26d4a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:44:54.974244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e70038f6-e105-478a-9719-17f48da26d4a admin admin] The token cache is being invalidate because project 5d2fa8874dab4d3bbc87161b540ffae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:44:54.975152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e70038f6-e105-478a-9719-17f48da26d4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:44:54.976006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e70038f6-e105-478a-9719-17f48da26d4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:44:54.981185 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1208/2391] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:44:54 2022] DELETE /identity/v3/projects/5d2fa8874dab4d3bbc87161b540ffae9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:45:04.548197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:45:04.964940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=23,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:45:15.122849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:45:15.124382 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:45:15.183639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848cb5e5-133f-40ae-85dc-f2b1aa0864e7 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:45:15.236079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1629bc5-0e7e-4994-a5c1-309be8aae36e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:45:15.236733 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1629bc5-0e7e-4994-a5c1-309be8aae36e service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:45:15.237017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1629bc5-0e7e-4994-a5c1-309be8aae36e service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:45:15.237299 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1629bc5-0e7e-4994-a5c1-309be8aae36e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:45:15.239126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e1629bc5-0e7e-4994-a5c1-309be8aae36e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:45:15.248329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1629bc5-0e7e-4994-a5c1-309be8aae36e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:45:15.249755 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1629bc5-0e7e-4994-a5c1-309be8aae36e service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:45:15.257739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1184/2392] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:45:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:45:17.306308 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343325-f059-4cec-bfd8-2c9bbabf6ddb None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:45:17.306797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343325-f059-4cec-bfd8-2c9bbabf6ddb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:45:17.306902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343325-f059-4cec-bfd8-2c9bbabf6ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:45:17.325869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-60343325-f059-4cec-bfd8-2c9bbabf6ddb None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:45:17.327120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-60343325-f059-4cec-bfd8-2c9bbabf6ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:45:17.384729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1209/2393] 10.222.0.60 () {58 vars in 967 bytes} [Mon Jul 25 12:45:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:45:17.407625 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1629bc5-0e7e-4994-a5c1-309be8aae36e service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:45:17.409067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e1629bc5-0e7e-4994-a5c1-309be8aae36e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:45:17.480658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1629bc5-0e7e-4994-a5c1-309be8aae36e service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:45:17.485987 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc49b3-e669-48c3-aafb-179a1167d3df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:45:17.486645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc49b3-e669-48c3-aafb-179a1167d3df service swift] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:45:17.486928 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc49b3-e669-48c3-aafb-179a1167d3df service swift] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:45:17.487205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc49b3-e669-48c3-aafb-179a1167d3df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:45:17.495799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc49b3-e669-48c3-aafb-179a1167d3df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:45:17.497387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc49b3-e669-48c3-aafb-179a1167d3df service swift] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:45:17.505523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1185/2394] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:45:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:45:27.378054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=25 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:45:27.502427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=78 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:45:31.742298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60343325-f059-4cec-bfd8-2c9bbabf6ddb None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:45:31.743885 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-60343325-f059-4cec-bfd8-2c9bbabf6ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:45:31.815649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60343325-f059-4cec-bfd8-2c9bbabf6ddb None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:45:31.821834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d011ece-7f20-4135-ab1a-e958e51fc6fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:45:31.822511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d011ece-7f20-4135-ab1a-e958e51fc6fd service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:45:31.822796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d011ece-7f20-4135-ab1a-e958e51fc6fd service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:45:31.823074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d011ece-7f20-4135-ab1a-e958e51fc6fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:45:31.824853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7d011ece-7f20-4135-ab1a-e958e51fc6fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:45:31.885851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d011ece-7f20-4135-ab1a-e958e51fc6fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:45:31.887673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d011ece-7f20-4135-ab1a-e958e51fc6fd service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:45:31.896673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1210/2395] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:45:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:45:41.892564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=56 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:45:56.613465 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f54fea-7484-454f-97c3-c4621f622f34 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:45:56.614092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f54fea-7484-454f-97c3-c4621f622f34 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:45:56.614092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f54fea-7484-454f-97c3-c4621f622f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:45:56.634887 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-05f54fea-7484-454f-97c3-c4621f622f34 None None] MFA Rules not processed for user `e56531920fc548a3a37f63ca0d1e5d82`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:45:56.636087 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-05f54fea-7484-454f-97c3-c4621f622f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:45:56.645150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1186/2396] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:45:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:46:03.303248 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d011ece-7f20-4135-ab1a-e958e51fc6fd service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:46:03.305176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7d011ece-7f20-4135-ab1a-e958e51fc6fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:03.321455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d011ece-7f20-4135-ab1a-e958e51fc6fd service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:46:03.327635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78563be-2ffb-4b33-a27e-47007c53cd5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:46:03.328321 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78563be-2ffb-4b33-a27e-47007c53cd5c service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:46:03.328618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78563be-2ffb-4b33-a27e-47007c53cd5c service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:46:03.328867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78563be-2ffb-4b33-a27e-47007c53cd5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:46:03.332034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d78563be-2ffb-4b33-a27e-47007c53cd5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:03.352656 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78563be-2ffb-4b33-a27e-47007c53cd5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:46:03.354381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78563be-2ffb-4b33-a27e-47007c53cd5c service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:46:03.365899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1211/2397] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:46:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:46:06.628980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:46:13.361122 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:46:21.478319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f54fea-7484-454f-97c3-c4621f622f34 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:46:21.492619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f54fea-7484-454f-97c3-c4621f622f34 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:46:21.498321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257a760-b8e4-4fe3-ba71-b03943ecd6ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:46:21.498967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257a760-b8e4-4fe3-ba71-b03943ecd6ca service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:46:21.499260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257a760-b8e4-4fe3-ba71-b03943ecd6ca service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:46:21.499527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257a760-b8e4-4fe3-ba71-b03943ecd6ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:46:21.501355 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4257a760-b8e4-4fe3-ba71-b03943ecd6ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:21.567942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257a760-b8e4-4fe3-ba71-b03943ecd6ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:46:21.570080 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257a760-b8e4-4fe3-ba71-b03943ecd6ca service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:46:21.579360 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1187/2398] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:46:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:46:31.575300 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=28 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:46:34.732096 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5c8ba-635b-405e-a3f7-b8562687df50 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:46:34.732598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5c8ba-635b-405e-a3f7-b8562687df50 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:46:34.732598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5c8ba-635b-405e-a3f7-b8562687df50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:46:34.751828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c2b5c8ba-635b-405e-a3f7-b8562687df50 None None] MFA Rules not processed for user `6f611c60491941ca8a0200212ab0c7a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:46:34.753034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c2b5c8ba-635b-405e-a3f7-b8562687df50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:34.759965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1212/2399] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:46:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:46:35.076650 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257a760-b8e4-4fe3-ba71-b03943ecd6ca service nova] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:46:35.078178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4257a760-b8e4-4fe3-ba71-b03943ecd6ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:35.090321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257a760-b8e4-4fe3-ba71-b03943ecd6ca service nova] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:46:35.094126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb5a66-159d-4450-890a-32af39068852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:46:35.094769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb5a66-159d-4450-890a-32af39068852 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:46:35.095048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb5a66-159d-4450-890a-32af39068852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:46:35.095322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb5a66-159d-4450-890a-32af39068852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:46:35.097017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c3cb5a66-159d-4450-890a-32af39068852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:35.106765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb5a66-159d-4450-890a-32af39068852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:46:35.108530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb5a66-159d-4450-890a-32af39068852 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:46:35.116986 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1188/2400] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:46:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:46:35.446939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b5c8ba-635b-405e-a3f7-b8562687df50 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:46:35.448343 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c2b5c8ba-635b-405e-a3f7-b8562687df50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:35.520664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b5c8ba-635b-405e-a3f7-b8562687df50 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:46:35.526164 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6699f-3cea-4c5a-b8f1-cc78d1241457 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:46:35.526811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6699f-3cea-4c5a-b8f1-cc78d1241457 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:46:35.527091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6699f-3cea-4c5a-b8f1-cc78d1241457 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:46:35.527356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6699f-3cea-4c5a-b8f1-cc78d1241457 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:46:35.529162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5ee6699f-3cea-4c5a-b8f1-cc78d1241457 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:35.540399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6699f-3cea-4c5a-b8f1-cc78d1241457 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:46:35.541994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6699f-3cea-4c5a-b8f1-cc78d1241457 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:46:35.552087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1213/2401] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:46:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:46:45.113719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=6 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:46:45.548727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=35 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:46:48.197426 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be883b5-039f-4bb2-8102-f12fb09d51d4 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:46:48.198166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be883b5-039f-4bb2-8102-f12fb09d51d4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:46:48.198166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be883b5-039f-4bb2-8102-f12fb09d51d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:46:48.218511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-2be883b5-039f-4bb2-8102-f12fb09d51d4 None None] MFA Rules not processed for user `1bc5799ac34349518854829ec2c259f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:46:48.219738 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2be883b5-039f-4bb2-8102-f12fb09d51d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:48.295514 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1189/2402] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:46:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:46:48.624873 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6699f-3cea-4c5a-b8f1-cc78d1241457 service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:46:48.626418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5ee6699f-3cea-4c5a-b8f1-cc78d1241457 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:48.638434 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6699f-3cea-4c5a-b8f1-cc78d1241457 service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:46:48.641893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf770026-62b0-4c68-b47b-a82df0d4e308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:46:48.642552 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf770026-62b0-4c68-b47b-a82df0d4e308 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:46:48.642833 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf770026-62b0-4c68-b47b-a82df0d4e308 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:46:48.643107 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf770026-62b0-4c68-b47b-a82df0d4e308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:46:48.644792 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bf770026-62b0-4c68-b47b-a82df0d4e308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:48.654936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf770026-62b0-4c68-b47b-a82df0d4e308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:46:48.656373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf770026-62b0-4c68-b47b-a82df0d4e308 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:46:48.664150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1214/2403] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:46:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:46:48.999609 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be883b5-039f-4bb2-8102-f12fb09d51d4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:46:49.023452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be883b5-039f-4bb2-8102-f12fb09d51d4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:46:49.034593 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b4703-f275-4372-9dc4-8568ba8de958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:46:49.035348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b4703-f275-4372-9dc4-8568ba8de958 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:46:49.035721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b4703-f275-4372-9dc4-8568ba8de958 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:46:49.036104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b4703-f275-4372-9dc4-8568ba8de958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:46:49.038124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-888b4703-f275-4372-9dc4-8568ba8de958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:49.064544 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b4703-f275-4372-9dc4-8568ba8de958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:46:49.066374 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b4703-f275-4372-9dc4-8568ba8de958 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:46:49.078737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1190/2404] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:46:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:46:53.150870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf770026-62b0-4c68-b47b-a82df0d4e308 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:46:53.165521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf770026-62b0-4c68-b47b-a82df0d4e308 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:46:53.172095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fb3a0-8617-4b72-b21a-5489bd0f3185 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:46:53.173027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fb3a0-8617-4b72-b21a-5489bd0f3185 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:46:53.173507 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fb3a0-8617-4b72-b21a-5489bd0f3185 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:46:53.173788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fb3a0-8617-4b72-b21a-5489bd0f3185 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:46:53.175675 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-741fb3a0-8617-4b72-b21a-5489bd0f3185 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:46:53.189502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741fb3a0-8617-4b72-b21a-5489bd0f3185 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:46:53.191169 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741fb3a0-8617-4b72-b21a-5489bd0f3185 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:46:53.204711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1215/2405] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:46:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:46:59.073835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=31 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:47:03.198962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=12 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:47:15.893087 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0bf80-b5c2-4f59-a7e4-7dadecb070a8 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:15.893635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0bf80-b5c2-4f59-a7e4-7dadecb070a8 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:15.893635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0bf80-b5c2-4f59-a7e4-7dadecb070a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:15.915983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-e1e0bf80-b5c2-4f59-a7e4-7dadecb070a8 None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:47:15.917126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e1e0bf80-b5c2-4f59-a7e4-7dadecb070a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:15.928096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1191/2406] 10.222.0.60 () {58 vars in 967 bytes} [Mon Jul 25 12:47:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:47:15.980851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fb3a0-8617-4b72-b21a-5489bd0f3185 service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:15.993367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fb3a0-8617-4b72-b21a-5489bd0f3185 service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:15.999265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398019d0-1771-401d-8302-3abcdbaa3ca3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:16.000004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398019d0-1771-401d-8302-3abcdbaa3ca3 service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:16.000325 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398019d0-1771-401d-8302-3abcdbaa3ca3 service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:16.000670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398019d0-1771-401d-8302-3abcdbaa3ca3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:16.022711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398019d0-1771-401d-8302-3abcdbaa3ca3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:16.022711 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398019d0-1771-401d-8302-3abcdbaa3ca3 service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:16.026494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1216/2407] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:47:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:47:19.406921 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0bf80-b5c2-4f59-a7e4-7dadecb070a8 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:19.417868 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0bf80-b5c2-4f59-a7e4-7dadecb070a8 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:19.422618 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84b441-5d71-45ed-86e6-10896640ada8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:19.423706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84b441-5d71-45ed-86e6-10896640ada8 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:19.424239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84b441-5d71-45ed-86e6-10896640ada8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:19.424599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84b441-5d71-45ed-86e6-10896640ada8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:19.426937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6b84b441-5d71-45ed-86e6-10896640ada8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:19.509835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84b441-5d71-45ed-86e6-10896640ada8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:19.511483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84b441-5d71-45ed-86e6-10896640ada8 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:19.523534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1192/2408] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:47:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:47:21.762605 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398019d0-1771-401d-8302-3abcdbaa3ca3 service swift] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:21.775204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398019d0-1771-401d-8302-3abcdbaa3ca3 service swift] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:21.782085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0774c-4049-4aa7-8281-6670541a7748 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:21.782759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0774c-4049-4aa7-8281-6670541a7748 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:21.783114 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0774c-4049-4aa7-8281-6670541a7748 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:21.783483 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0774c-4049-4aa7-8281-6670541a7748 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:21.785511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7eb0774c-4049-4aa7-8281-6670541a7748 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:21.800385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0774c-4049-4aa7-8281-6670541a7748 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:21.802166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0774c-4049-4aa7-8281-6670541a7748 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:21.814247 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1217/2409] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:47:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:47:25.409841 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84b441-5d71-45ed-86e6-10896640ada8 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:25.411545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6b84b441-5d71-45ed-86e6-10896640ada8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:25.424005 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84b441-5d71-45ed-86e6-10896640ada8 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:25.428842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:25.429666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:25.430524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:25.430821 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] PATH_INFO: `/v3/users/f5032f6d4f074cae838066344fcea598` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:25.432187 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5032f6d4f074cae838066344fcea598)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:25.434236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:25.484617 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5032f6d4f074cae838066344fcea598'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.485314 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5032f6d4f074cae838066344fcea598'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.486034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5032f6d4f074cae838066344fcea598'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.493966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5032f6d4f074cae838066344fcea598'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.501773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5032f6d4f074cae838066344fcea598'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.521857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1193/2410] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:47:25 2022] DELETE /identity/v3/users/f5032f6d4f074cae838066344fcea598 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:47:25.699496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0774c-4049-4aa7-8281-6670541a7748 service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:25.710687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0774c-4049-4aa7-8281-6670541a7748 service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:25.771683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:25.772555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:25.772840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:25.773118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] PATH_INFO: `/v3/projects/8260111bf70a4afab166caf5a4881931` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:25.774864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8260111bf70a4afab166caf5a4881931)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:25.776841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:25.823685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8260111bf70a4afab166caf5a4881931', 'request_id': 'req-4361e54f-d64b-4045-b40a-8aed6da928f0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.830871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8260111bf70a4afab166caf5a4881931', 'request_id': 'req-4361e54f-d64b-4045-b40a-8aed6da928f0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.838025 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8260111bf70a4afab166caf5a4881931', 'request_id': 'req-4361e54f-d64b-4045-b40a-8aed6da928f0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.848115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] The token cache is being invalidate because project 8260111bf70a4afab166caf5a4881931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:47:25.848398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.849032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.850600 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1218/2411] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:47:25 2022] DELETE /identity/v3/projects/8260111bf70a4afab166caf5a4881931 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:47:25.854836 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:25.856265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:25.869512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b971789-0cfa-4bf1-ba9c-1db0d378addf admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:25.924884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:25.925753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:25.926085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:25.926357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] PATH_INFO: `/v3/users/8fdc221326054df385a450b89afbe8ac` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:25.927559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fdc221326054df385a450b89afbe8ac)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:25.929488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:25.931095 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:47:25.986857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fdc221326054df385a450b89afbe8ac'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.987605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fdc221326054df385a450b89afbe8ac'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.988225 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fdc221326054df385a450b89afbe8ac'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:25.997755 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fdc221326054df385a450b89afbe8ac'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:26.006574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fdc221326054df385a450b89afbe8ac'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:26.027247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1194/2412] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:47:25 2022] DELETE /identity/v3/users/8fdc221326054df385a450b89afbe8ac => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:47:26.294772 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:26.296328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:26.309845 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361e54f-d64b-4045-b40a-8aed6da928f0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:26.370855 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:26.371720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:26.372004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:26.372283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] PATH_INFO: `/v3/projects/84f6a81ab18042898f1762057c006de3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:26.373479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f6a81ab18042898f1762057c006de3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:26.375471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:26.381762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:47:26.423306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f6a81ab18042898f1762057c006de3', 'request_id': 'req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:26.431361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f6a81ab18042898f1762057c006de3', 'request_id': 'req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:26.437702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f6a81ab18042898f1762057c006de3', 'request_id': 'req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:26.447981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] The token cache is being invalidate because project 84f6a81ab18042898f1762057c006de3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:47:26.448268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:26.448985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:26.450491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1219/2413] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:47:26 2022] DELETE /identity/v3/projects/84f6a81ab18042898f1762057c006de3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:47:28.590244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:28.591657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:28.680766 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b95d8-3d77-409a-aa56-680e2612cc85 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:28.733760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431c682-df08-4b63-8aa1-6897b18926ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:28.734457 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431c682-df08-4b63-8aa1-6897b18926ff service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:28.734738 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431c682-df08-4b63-8aa1-6897b18926ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:28.735010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431c682-df08-4b63-8aa1-6897b18926ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:28.736842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2431c682-df08-4b63-8aa1-6897b18926ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:28.759684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:47:28.848058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431c682-df08-4b63-8aa1-6897b18926ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:28.849439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431c682-df08-4b63-8aa1-6897b18926ff service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:28.869727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1195/2414] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:47:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:47:32.027013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:32.028542 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:32.100924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f06e9-11ae-4a5e-b2da-290ca3f4f20c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:32.106290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885e16e-8082-4977-b717-b161e03ade72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:32.107161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885e16e-8082-4977-b717-b161e03ade72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:32.107683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885e16e-8082-4977-b717-b161e03ade72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:32.107963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885e16e-8082-4977-b717-b161e03ade72 admin admin] PATH_INFO: `/v3/users/e56531920fc548a3a37f63ca0d1e5d82` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:32.109153 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885e16e-8082-4977-b717-b161e03ade72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e56531920fc548a3a37f63ca0d1e5d82)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:32.111113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885e16e-8082-4977-b717-b161e03ade72 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:32.157142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d885e16e-8082-4977-b717-b161e03ade72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e56531920fc548a3a37f63ca0d1e5d82'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:32.163663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d885e16e-8082-4977-b717-b161e03ade72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e56531920fc548a3a37f63ca0d1e5d82'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:32.165149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d885e16e-8082-4977-b717-b161e03ade72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e56531920fc548a3a37f63ca0d1e5d82'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:32.172351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d885e16e-8082-4977-b717-b161e03ade72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e56531920fc548a3a37f63ca0d1e5d82'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:32.173065 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d885e16e-8082-4977-b717-b161e03ade72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e56531920fc548a3a37f63ca0d1e5d82'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:32.192543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1220/2415] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:47:32 2022] DELETE /identity/v3/users/e56531920fc548a3a37f63ca0d1e5d82 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:47:32.378607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431c682-df08-4b63-8aa1-6897b18926ff service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:32.388748 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431c682-df08-4b63-8aa1-6897b18926ff service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:32.451755 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c9938-f727-4995-b079-2f2b295abd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:32.452607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c9938-f727-4995-b079-2f2b295abd7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:32.452885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c9938-f727-4995-b079-2f2b295abd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:32.453153 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c9938-f727-4995-b079-2f2b295abd7e admin admin] PATH_INFO: `/v3/projects/3b0ca10639aa47dea81db9799273acba` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:32.454368 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c9938-f727-4995-b079-2f2b295abd7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0ca10639aa47dea81db9799273acba)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:32.456271 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c9938-f727-4995-b079-2f2b295abd7e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:32.503174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f63c9938-f727-4995-b079-2f2b295abd7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0ca10639aa47dea81db9799273acba', 'request_id': 'req-f63c9938-f727-4995-b079-2f2b295abd7e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:32.509252 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f63c9938-f727-4995-b079-2f2b295abd7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0ca10639aa47dea81db9799273acba', 'request_id': 'req-f63c9938-f727-4995-b079-2f2b295abd7e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:32.514709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f63c9938-f727-4995-b079-2f2b295abd7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0ca10639aa47dea81db9799273acba', 'request_id': 'req-f63c9938-f727-4995-b079-2f2b295abd7e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:32.522479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:47:32.528406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f63c9938-f727-4995-b079-2f2b295abd7e admin admin] The token cache is being invalidate because project 3b0ca10639aa47dea81db9799273acba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:47:32.528909 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f63c9938-f727-4995-b079-2f2b295abd7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:32.529594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f63c9938-f727-4995-b079-2f2b295abd7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:47:32.531594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1196/2416] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:47:32 2022] DELETE /identity/v3/projects/3b0ca10639aa47dea81db9799273acba => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:47:32.537486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fbffa-6b5c-47fe-9bf6-fce4dead1f82 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:32.537771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fbffa-6b5c-47fe-9bf6-fce4dead1f82 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:32.538077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fbffa-6b5c-47fe-9bf6-fce4dead1f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:32.555392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:47:32.598903 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-fe2fbffa-6b5c-47fe-9bf6-fce4dead1f82 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:47:32.600109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fe2fbffa-6b5c-47fe-9bf6-fce4dead1f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:32.662887 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1221/2417] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:47:32.671799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b8d4c-405c-4750-9ce3-3bc02929b62f None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:32.672089 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b8d4c-405c-4750-9ce3-3bc02929b62f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:32.672365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b8d4c-405c-4750-9ce3-3bc02929b62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:32.692881 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-515b8d4c-405c-4750-9ce3-3bc02929b62f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:47:32.694124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-515b8d4c-405c-4750-9ce3-3bc02929b62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:32.702703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1197/2418] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:47:32.707113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fbffa-6b5c-47fe-9bf6-fce4dead1f82 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:32.717229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fbffa-6b5c-47fe-9bf6-fce4dead1f82 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:32.722077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b34fc-1c3d-421f-8215-a9b15177f54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:32.722743 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b34fc-1c3d-421f-8215-a9b15177f54c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:32.723022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b34fc-1c3d-421f-8215-a9b15177f54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:32.723294 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b34fc-1c3d-421f-8215-a9b15177f54c admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:32.724022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b34fc-1c3d-421f-8215-a9b15177f54c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:47:32.724324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b34fc-1c3d-421f-8215-a9b15177f54c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:32.725952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b34fc-1c3d-421f-8215-a9b15177f54c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:32.738449 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1222/2419] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:47:32.742886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b8d4c-405c-4750-9ce3-3bc02929b62f None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:32.752481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b8d4c-405c-4750-9ce3-3bc02929b62f None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:32.757158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087cc80-f41a-4950-b160-5b9272e0b7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:32.758367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087cc80-f41a-4950-b160-5b9272e0b7d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:32.758655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087cc80-f41a-4950-b160-5b9272e0b7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:32.758933 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087cc80-f41a-4950-b160-5b9272e0b7d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:32.759639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087cc80-f41a-4950-b160-5b9272e0b7d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:32.761549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087cc80-f41a-4950-b160-5b9272e0b7d3 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:32.790581 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1198/2420] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:47:32 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:47:32.794881 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b34fc-1c3d-421f-8215-a9b15177f54c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:32.805033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b34fc-1c3d-421f-8215-a9b15177f54c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:32.861488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78f270-076d-45c6-a4d3-7c8d8efe508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:32.862193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78f270-076d-45c6-a4d3-7c8d8efe508d admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:32.862477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78f270-076d-45c6-a4d3-7c8d8efe508d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:32.862755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78f270-076d-45c6-a4d3-7c8d8efe508d admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:32.863451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78f270-076d-45c6-a4d3-7c8d8efe508d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:32.865387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78f270-076d-45c6-a4d3-7c8d8efe508d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:32.887639 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1223/2421] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:47:32 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:47:32.891843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087cc80-f41a-4950-b160-5b9272e0b7d3 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:32.902664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087cc80-f41a-4950-b160-5b9272e0b7d3 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:32.907278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afa978-2219-4738-b0da-2c2426b43e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:32.907935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afa978-2219-4738-b0da-2c2426b43e80 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:32.908217 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afa978-2219-4738-b0da-2c2426b43e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:32.908492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afa978-2219-4738-b0da-2c2426b43e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:32.909241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91afa978-2219-4738-b0da-2c2426b43e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:32.910909 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91afa978-2219-4738-b0da-2c2426b43e80 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:32.921181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1199/2422] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:47:32.925265 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78f270-076d-45c6-a4d3-7c8d8efe508d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:32.935803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78f270-076d-45c6-a4d3-7c8d8efe508d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:32.940399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ff28d-d77e-4eb2-ba80-fa72a6031e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:32.941061 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ff28d-d77e-4eb2-ba80-fa72a6031e52 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:32.941428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ff28d-d77e-4eb2-ba80-fa72a6031e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:32.941647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ff28d-d77e-4eb2-ba80-fa72a6031e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:32.942485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ff28d-d77e-4eb2-ba80-fa72a6031e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:32.944086 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ff28d-d77e-4eb2-ba80-fa72a6031e52 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:32.955401 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1224/2423] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:47:32.959562 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afa978-2219-4738-b0da-2c2426b43e80 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:32.969604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afa978-2219-4738-b0da-2c2426b43e80 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:32.974522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600d8c8-00b4-4010-8128-6d8ec3f67d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:32.975403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600d8c8-00b4-4010-8128-6d8ec3f67d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:32.975677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600d8c8-00b4-4010-8128-6d8ec3f67d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:32.975948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600d8c8-00b4-4010-8128-6d8ec3f67d9e admin admin] PATH_INFO: `/v3/projects/47ad60e5634942149abd3837dc89a019/users/9a4b1f0404e74e0e9e53e25ad92dcfed/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:32.990776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600d8c8-00b4-4010-8128-6d8ec3f67d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ad60e5634942149abd3837dc89a019, user_id=9a4b1f0404e74e0e9e53e25ad92dcfed, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:32.994022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600d8c8-00b4-4010-8128-6d8ec3f67d9e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:33.007038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1200/2424] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:47:32 2022] PUT /identity/v3/projects/47ad60e5634942149abd3837dc89a019/users/9a4b1f0404e74e0e9e53e25ad92dcfed/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:47:33.012744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a96c0-952c-44e2-a8a4-9d357ee8ab7d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:33.013036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a96c0-952c-44e2-a8a4-9d357ee8ab7d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:33.013315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a96c0-952c-44e2-a8a4-9d357ee8ab7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:33.032030 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-be2a96c0-952c-44e2-a8a4-9d357ee8ab7d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:47:33.033223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-be2a96c0-952c-44e2-a8a4-9d357ee8ab7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:33.078409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:47:33.094893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1225/2425] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:47:33.105720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600d8c8-00b4-4010-8128-6d8ec3f67d9e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:33.107754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d600d8c8-00b4-4010-8128-6d8ec3f67d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:33.164503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600d8c8-00b4-4010-8128-6d8ec3f67d9e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:33.168136 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e0887-5126-4837-988d-252612b945ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:33.168788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e0887-5126-4837-988d-252612b945ff service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:33.169068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e0887-5126-4837-988d-252612b945ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:33.169363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e0887-5126-4837-988d-252612b945ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:33.179086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e0887-5126-4837-988d-252612b945ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:33.181040 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e0887-5126-4837-988d-252612b945ff service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:33.191352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1201/2426] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:47:37.413086 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0a8cb-0d0f-4afa-9143-adcbd9a605be None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:37.413620 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0a8cb-0d0f-4afa-9143-adcbd9a605be None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:37.413730 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0a8cb-0d0f-4afa-9143-adcbd9a605be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:37.485715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-65a0a8cb-0d0f-4afa-9143-adcbd9a605be None None] MFA Rules not processed for user `9a4b1f0404e74e0e9e53e25ad92dcfed`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:47:37.487617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-65a0a8cb-0d0f-4afa-9143-adcbd9a605be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:37.590847 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1226/2427] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:47:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:47:37.606867 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e0887-5126-4837-988d-252612b945ff service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:47:37.608989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e11e0887-5126-4837-988d-252612b945ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:37.749650 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e0887-5126-4837-988d-252612b945ff service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:47:37.763509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25e803-ce5c-4522-bdf4-b8db5f0ae0f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:47:37.764138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25e803-ce5c-4522-bdf4-b8db5f0ae0f1 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:37.764418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25e803-ce5c-4522-bdf4-b8db5f0ae0f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:37.764691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25e803-ce5c-4522-bdf4-b8db5f0ae0f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:37.771383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:47:37.774720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25e803-ce5c-4522-bdf4-b8db5f0ae0f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:47:37.776940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25e803-ce5c-4522-bdf4-b8db5f0ae0f1 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:47:37.790202 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1202/2428] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:47:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:47:39.330951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a058397-f18b-4c3d-9991-2419e166943d None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:47:39.331536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a058397-f18b-4c3d-9991-2419e166943d None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:47:39.331536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a058397-f18b-4c3d-9991-2419e166943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:47:39.355505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2a058397-f18b-4c3d-9991-2419e166943d None None] MFA Rules not processed for user `9a4b1f0404e74e0e9e53e25ad92dcfed`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:47:39.357000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2a058397-f18b-4c3d-9991-2419e166943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:47:39.366556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1227/2429] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:47:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:47:47.787967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:47:49.348639 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=40 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:48:37.055998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25e803-ce5c-4522-bdf4-b8db5f0ae0f1 service nova] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:48:37.057482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-dd25e803-ce5c-4522-bdf4-b8db5f0ae0f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:48:37.155074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25e803-ce5c-4522-bdf4-b8db5f0ae0f1 service nova] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:48:37.164057 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6092d0-46d8-430f-a98f-e9b1d5512010 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:48:37.164728 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6092d0-46d8-430f-a98f-e9b1d5512010 service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:48:37.165310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6092d0-46d8-430f-a98f-e9b1d5512010 service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:48:37.165761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6092d0-46d8-430f-a98f-e9b1d5512010 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:48:37.172241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-db6092d0-46d8-430f-a98f-e9b1d5512010 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:48:37.187234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6092d0-46d8-430f-a98f-e9b1d5512010 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:48:37.189694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6092d0-46d8-430f-a98f-e9b1d5512010 service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:48:37.201391 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1203/2430] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:48:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:48:40.304221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8b163-287f-4c40-a5ed-7e6f042ee148 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:48:40.304803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8b163-287f-4c40-a5ed-7e6f042ee148 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:48:40.304803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8b163-287f-4c40-a5ed-7e6f042ee148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:48:40.324478 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-79d8b163-287f-4c40-a5ed-7e6f042ee148 None None] MFA Rules not processed for user `6f611c60491941ca8a0200212ab0c7a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:48:40.325673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-79d8b163-287f-4c40-a5ed-7e6f042ee148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:48:40.392781 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1228/2431] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:48:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:48:47.196692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=41 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:48:50.385614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=25 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:48:57.964346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db2461-206d-45e1-9d7f-0e5c1f42e65e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:48:57.964946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db2461-206d-45e1-9d7f-0e5c1f42e65e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:48:57.964946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db2461-206d-45e1-9d7f-0e5c1f42e65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:48:57.997667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-00db2461-206d-45e1-9d7f-0e5c1f42e65e None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:48:57.999002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-00db2461-206d-45e1-9d7f-0e5c1f42e65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:48:58.058372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1204/2432] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:48:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:48:58.107912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8b163-287f-4c40-a5ed-7e6f042ee148 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:48:58.109358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-79d8b163-287f-4c40-a5ed-7e6f042ee148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:48:58.183660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8b163-287f-4c40-a5ed-7e6f042ee148 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:48:58.189169 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ff0b5-88bf-4283-9295-9fdeda325792 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:48:58.189841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ff0b5-88bf-4283-9295-9fdeda325792 service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:48:58.190153 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ff0b5-88bf-4283-9295-9fdeda325792 service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:48:58.190429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ff0b5-88bf-4283-9295-9fdeda325792 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:48:58.199394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ff0b5-88bf-4283-9295-9fdeda325792 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:48:58.200999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ff0b5-88bf-4283-9295-9fdeda325792 service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:48:58.209398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1229/2433] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:48:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:49:08.050939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=29 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:49:08.206016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=32 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:49:15.343043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfb9e6-2760-4014-b9bf-75f41dcb41f3 None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:15.343971 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfb9e6-2760-4014-b9bf-75f41dcb41f3 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:15.344275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfb9e6-2760-4014-b9bf-75f41dcb41f3 None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:15.345883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1205/2434] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:49:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:49:15.352959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5ba8c-9f6e-42c6-b255-b5b2c5d26a69 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:15.353451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5ba8c-9f6e-42c6-b255-b5b2c5d26a69 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:15.353663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5ba8c-9f6e-42c6-b255-b5b2c5d26a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:15.374518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-47a5ba8c-9f6e-42c6-b255-b5b2c5d26a69 None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:49:15.375759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-47a5ba8c-9f6e-42c6-b255-b5b2c5d26a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:49:15.385728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1230/2435] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:49:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:49:15.399064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfb9e6-2760-4014-b9bf-75f41dcb41f3 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:49:15.400769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fdcfb9e6-2760-4014-b9bf-75f41dcb41f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:49:15.469468 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfb9e6-2760-4014-b9bf-75f41dcb41f3 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:49:15.474078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cc1c2-a71e-46b3-b200-f86b27976baf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b07dcd8c0de4d1694d4363520726df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:49:15.475419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cc1c2-a71e-46b3-b200-f86b27976baf service glance] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:15.475756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cc1c2-a71e-46b3-b200-f86b27976baf service glance] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:15.476035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cc1c2-a71e-46b3-b200-f86b27976baf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:15.491575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cc1c2-a71e-46b3-b200-f86b27976baf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:49:15.493663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cc1c2-a71e-46b3-b200-f86b27976baf service glance] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:49:15.505126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1206/2436] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:49:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:49:15.601457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb6833-2ede-4011-813b-c0ae87dbaa13 None None] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:15.601748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb6833-2ede-4011-813b-c0ae87dbaa13 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:15.602053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb6833-2ede-4011-813b-c0ae87dbaa13 None None] PATH_INFO: `/` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:15.603591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1231/2437] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:49:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:49:15.610081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4e245-f2f4-4d1a-8c98-d473e51885a1 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:15.610504 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4e245-f2f4-4d1a-8c98-d473e51885a1 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:15.610834 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4e245-f2f4-4d1a-8c98-d473e51885a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:15.645759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-31f4e245-f2f4-4d1a-8c98-d473e51885a1 None None] MFA Rules not processed for user `375d321d6de2494794254e50151c02c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:49:15.647667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-31f4e245-f2f4-4d1a-8c98-d473e51885a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:49:15.665314 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1207/2438] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:49:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:49:15.677804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb6833-2ede-4011-813b-c0ae87dbaa13 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:49:15.679752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cefb6833-2ede-4011-813b-c0ae87dbaa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:49:15.694108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb6833-2ede-4011-813b-c0ae87dbaa13 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:49:15.698040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29ec5c-d58d-4ea9-bba0-22e8a57a8b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:49:15.698811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29ec5c-d58d-4ea9-bba0-22e8a57a8b87 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:15.699123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29ec5c-d58d-4ea9-bba0-22e8a57a8b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:15.699435 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29ec5c-d58d-4ea9-bba0-22e8a57a8b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:15.707652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29ec5c-d58d-4ea9-bba0-22e8a57a8b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:49:15.709721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29ec5c-d58d-4ea9-bba0-22e8a57a8b87 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:49:15.718127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1232/2439] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:49:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:49:15.814304 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e5e27-c3ca-4f00-a0c0-e5a40298a3cd None None] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:15.814722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e5e27-c3ca-4f00-a0c0-e5a40298a3cd None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:15.815103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e5e27-c3ca-4f00-a0c0-e5a40298a3cd None None] PATH_INFO: `/` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:15.816581 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1208/2440] 10.222.0.60 () {54 vars in 921 bytes} [Mon Jul 25 12:49:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:49:15.822747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652fe42-7ab2-4ccf-be56-20f412920e94 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:15.823040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652fe42-7ab2-4ccf-be56-20f412920e94 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:15.823322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652fe42-7ab2-4ccf-be56-20f412920e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:15.844700 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a652fe42-7ab2-4ccf-be56-20f412920e94 None None] MFA Rules not processed for user `ccf2dedaf7a449d2b51b9e9f3b420091`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:49:15.845968 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a652fe42-7ab2-4ccf-be56-20f412920e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:49:15.856032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1233/2441] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Jul 25 12:49:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:49:15.867036 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e5e27-c3ca-4f00-a0c0-e5a40298a3cd None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:49:15.877934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e5e27-c3ca-4f00-a0c0-e5a40298a3cd None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:49:15.881470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d69825-7c7c-4650-b950-91a767bb2c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:49:15.882161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d69825-7c7c-4650-b950-91a767bb2c7e service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:15.882448 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d69825-7c7c-4650-b950-91a767bb2c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:15.882724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d69825-7c7c-4650-b950-91a767bb2c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:15.894778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d69825-7c7c-4650-b950-91a767bb2c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:49:15.896809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d69825-7c7c-4650-b950-91a767bb2c7e service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:49:15.908343 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1209/2442] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:49:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:49:17.707367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a31098-db35-4553-9301-e4e840da1fb2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:17.707871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a31098-db35-4553-9301-e4e840da1fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:17.707972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a31098-db35-4553-9301-e4e840da1fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:17.734759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-94a31098-db35-4553-9301-e4e840da1fb2 None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:49:17.735984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-94a31098-db35-4553-9301-e4e840da1fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:49:17.743416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1234/2443] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:49:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:49:17.757267 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d69825-7c7c-4650-b950-91a767bb2c7e service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:49:17.768915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d69825-7c7c-4650-b950-91a767bb2c7e service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:49:17.774703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a93787-30db-4b46-8b2e-8819c85f93ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:49:17.775369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a93787-30db-4b46-8b2e-8819c85f93ab service swift] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:17.775649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a93787-30db-4b46-8b2e-8819c85f93ab service swift] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:17.775929 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a93787-30db-4b46-8b2e-8819c85f93ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:17.795319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a93787-30db-4b46-8b2e-8819c85f93ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:49:17.798376 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a93787-30db-4b46-8b2e-8819c85f93ab service swift] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:49:17.810450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1210/2444] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:49:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:49:27.729517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=15 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:49:27.808361 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=50 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:49:34.185312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a31098-db35-4553-9301-e4e840da1fb2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:49:34.186802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-94a31098-db35-4553-9301-e4e840da1fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:49:34.275245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a31098-db35-4553-9301-e4e840da1fb2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:49:34.279053 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc247c-3e9d-4a97-a435-e995aabba4b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:49:34.279806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc247c-3e9d-4a97-a435-e995aabba4b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:34.280117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc247c-3e9d-4a97-a435-e995aabba4b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:34.280403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc247c-3e9d-4a97-a435-e995aabba4b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:34.282280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-26dc247c-3e9d-4a97-a435-e995aabba4b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:49:34.296565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc247c-3e9d-4a97-a435-e995aabba4b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:49:34.298033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc247c-3e9d-4a97-a435-e995aabba4b1 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:49:34.309774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1235/2445] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:49:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:49:36.738577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a93787-30db-4b46-8b2e-8819c85f93ab service swift] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:49:36.747903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a93787-30db-4b46-8b2e-8819c85f93ab service swift] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:49:36.751520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73586bc0-7f0b-4b66-b708-45ba31646577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:49:36.752158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73586bc0-7f0b-4b66-b708-45ba31646577 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:36.752444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73586bc0-7f0b-4b66-b708-45ba31646577 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:36.752723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73586bc0-7f0b-4b66-b708-45ba31646577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:36.754601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-73586bc0-7f0b-4b66-b708-45ba31646577 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:49:36.767891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73586bc0-7f0b-4b66-b708-45ba31646577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:49:36.769312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73586bc0-7f0b-4b66-b708-45ba31646577 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:49:36.781323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1211/2446] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:49:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:49:39.137275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc247c-3e9d-4a97-a435-e995aabba4b1 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:49:39.148769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc247c-3e9d-4a97-a435-e995aabba4b1 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:49:39.154451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45a6a4-76bb-4e2c-9afa-fb03a5d76283 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:49:39.155084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a6a4-76bb-4e2c-9afa-fb03a5d76283 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:39.155363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a6a4-76bb-4e2c-9afa-fb03a5d76283 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:39.155632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a6a4-76bb-4e2c-9afa-fb03a5d76283 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:39.157384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bd45a6a4-76bb-4e2c-9afa-fb03a5d76283 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:49:39.245300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45a6a4-76bb-4e2c-9afa-fb03a5d76283 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:49:39.247034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45a6a4-76bb-4e2c-9afa-fb03a5d76283 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:49:39.266348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1236/2447] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:49:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:49:39.945150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f5375-2742-4ce7-893d-bc57f849587f None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:39.945626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f5375-2742-4ce7-893d-bc57f849587f None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:39.945749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f5375-2742-4ce7-893d-bc57f849587f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:39.965425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-140f5375-2742-4ce7-893d-bc57f849587f None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:49:39.966686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-140f5375-2742-4ce7-893d-bc57f849587f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:49:39.977951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1212/2448] 10.222.0.60 () {58 vars in 967 bytes} [Mon Jul 25 12:49:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:49:39.993102 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45a6a4-76bb-4e2c-9afa-fb03a5d76283 service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:49:40.005049 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45a6a4-76bb-4e2c-9afa-fb03a5d76283 service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:49:40.011092 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b68be1-7fb9-4548-a8f3-3325941d2f64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:49:40.011770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b68be1-7fb9-4548-a8f3-3325941d2f64 service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:49:40.012054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b68be1-7fb9-4548-a8f3-3325941d2f64 service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:49:40.012333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b68be1-7fb9-4548-a8f3-3325941d2f64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:49:40.022207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b68be1-7fb9-4548-a8f3-3325941d2f64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:49:40.023818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b68be1-7fb9-4548-a8f3-3325941d2f64 service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:49:40.032587 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1237/2449] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:49:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:49:49.959734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=9 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:49:50.029038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=82 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:50:43.159104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f5375-2742-4ce7-893d-bc57f849587f None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:43.168744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f5375-2742-4ce7-893d-bc57f849587f None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:43.172451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01145c-95a9-4ca7-93f8-9fc76c0f647e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:43.173105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01145c-95a9-4ca7-93f8-9fc76c0f647e service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:43.173384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01145c-95a9-4ca7-93f8-9fc76c0f647e service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:43.173655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01145c-95a9-4ca7-93f8-9fc76c0f647e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:43.175487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-3e01145c-95a9-4ca7-93f8-9fc76c0f647e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:43.185601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01145c-95a9-4ca7-93f8-9fc76c0f647e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:43.187088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01145c-95a9-4ca7-93f8-9fc76c0f647e service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:43.195790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1213/2450] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:50:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:50:44.609876 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b68be1-7fb9-4548-a8f3-3325941d2f64 service swift] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:44.620977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b68be1-7fb9-4548-a8f3-3325941d2f64 service swift] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:44.625055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a3165-e564-4e54-847b-387a16faa21b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:44.625717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3165-e564-4e54-847b-387a16faa21b service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:44.626043 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3165-e564-4e54-847b-387a16faa21b service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:44.626324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3165-e564-4e54-847b-387a16faa21b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:44.628159 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ee1a3165-e564-4e54-847b-387a16faa21b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:44.642864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a3165-e564-4e54-847b-387a16faa21b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:44.644338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a3165-e564-4e54-847b-387a16faa21b service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:44.654918 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1238/2451] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:50:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:50:47.947716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01145c-95a9-4ca7-93f8-9fc76c0f647e service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:47.960550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01145c-95a9-4ca7-93f8-9fc76c0f647e service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:47.968222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45030856-b17c-48a6-840e-b39fa7fa5f04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:47.968983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45030856-b17c-48a6-840e-b39fa7fa5f04 service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:47.969333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45030856-b17c-48a6-840e-b39fa7fa5f04 service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:47.969693 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45030856-b17c-48a6-840e-b39fa7fa5f04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:47.971586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-45030856-b17c-48a6-840e-b39fa7fa5f04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:47.984452 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45030856-b17c-48a6-840e-b39fa7fa5f04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:47.986218 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45030856-b17c-48a6-840e-b39fa7fa5f04 service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:47.998565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1214/2452] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:50:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:50:48.021050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a3165-e564-4e54-847b-387a16faa21b service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:48.022455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ee1a3165-e564-4e54-847b-387a16faa21b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:48.036263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a3165-e564-4e54-847b-387a16faa21b service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:48.041249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268cc5f-3f66-4d7f-b674-a397330a06ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:48.042155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268cc5f-3f66-4d7f-b674-a397330a06ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:48.042431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268cc5f-3f66-4d7f-b674-a397330a06ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:48.042705 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268cc5f-3f66-4d7f-b674-a397330a06ac admin admin] PATH_INFO: `/v3/users/6f611c60491941ca8a0200212ab0c7a7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:48.058837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268cc5f-3f66-4d7f-b674-a397330a06ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f611c60491941ca8a0200212ab0c7a7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:48.060794 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268cc5f-3f66-4d7f-b674-a397330a06ac admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:48.115692 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6268cc5f-3f66-4d7f-b674-a397330a06ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f611c60491941ca8a0200212ab0c7a7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:48.129720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6268cc5f-3f66-4d7f-b674-a397330a06ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f611c60491941ca8a0200212ab0c7a7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:48.130410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6268cc5f-3f66-4d7f-b674-a397330a06ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f611c60491941ca8a0200212ab0c7a7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:48.137642 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6268cc5f-3f66-4d7f-b674-a397330a06ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f611c60491941ca8a0200212ab0c7a7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:48.138342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-6268cc5f-3f66-4d7f-b674-a397330a06ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f611c60491941ca8a0200212ab0c7a7'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:48.156041 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1239/2453] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:50:48 2022] DELETE /identity/v3/users/6f611c60491941ca8a0200212ab0c7a7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:50:48.333477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45030856-b17c-48a6-840e-b39fa7fa5f04 service placement] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:48.344951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45030856-b17c-48a6-840e-b39fa7fa5f04 service placement] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:48.408439 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:48.409344 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:48.409685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:48.409990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45 admin admin] PATH_INFO: `/v3/projects/49256bcb551a4f8680db9a73763c435e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:48.422352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49256bcb551a4f8680db9a73763c435e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:48.424295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:48.476833 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49256bcb551a4f8680db9a73763c435e', 'request_id': 'req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:48.484361 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49256bcb551a4f8680db9a73763c435e', 'request_id': 'req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:48.491550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49256bcb551a4f8680db9a73763c435e', 'request_id': 'req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:48.507843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45 admin admin] The token cache is being invalidate because project 49256bcb551a4f8680db9a73763c435e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:50:48.508134 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:48.508756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-b97b8dd7-9bb5-4cdc-8f32-d9a768bc3f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:48.510337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1215/2454] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:50:48 2022] DELETE /identity/v3/projects/49256bcb551a4f8680db9a73763c435e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:50:48.515442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd579734-3e71-4552-bbe3-6a81bf578ccf None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:48.515734 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd579734-3e71-4552-bbe3-6a81bf578ccf None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:48.516017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd579734-3e71-4552-bbe3-6a81bf578ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:48.536638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-cd579734-3e71-4552-bbe3-6a81bf578ccf None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:50:48.537884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cd579734-3e71-4552-bbe3-6a81bf578ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:48.608962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1240/2455] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:50:48.617140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72854a-7e19-4b43-b642-2c9707963c69 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:48.617429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72854a-7e19-4b43-b642-2c9707963c69 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:48.617708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72854a-7e19-4b43-b642-2c9707963c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:48.639310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-3f72854a-7e19-4b43-b642-2c9707963c69 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:50:48.640553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-3f72854a-7e19-4b43-b642-2c9707963c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:48.649958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1216/2456] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:50:48.654231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd579734-3e71-4552-bbe3-6a81bf578ccf None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:48.664884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd579734-3e71-4552-bbe3-6a81bf578ccf None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:48.670015 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36cd45-52fe-489a-a67c-3284e9ee1d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:48.670690 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36cd45-52fe-489a-a67c-3284e9ee1d22 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:48.670976 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36cd45-52fe-489a-a67c-3284e9ee1d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:48.671254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36cd45-52fe-489a-a67c-3284e9ee1d22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:48.672001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36cd45-52fe-489a-a67c-3284e9ee1d22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:50:48.672308 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36cd45-52fe-489a-a67c-3284e9ee1d22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:48.674111 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36cd45-52fe-489a-a67c-3284e9ee1d22 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:48.687452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1241/2457] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:50:48.691817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72854a-7e19-4b43-b642-2c9707963c69 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:48.704716 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72854a-7e19-4b43-b642-2c9707963c69 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:48.710582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c66702-feb4-457a-9b15-7abb9d9455f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:48.711331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c66702-feb4-457a-9b15-7abb9d9455f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:48.711596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c66702-feb4-457a-9b15-7abb9d9455f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:48.711936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c66702-feb4-457a-9b15-7abb9d9455f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:48.712756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c66702-feb4-457a-9b15-7abb9d9455f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:48.714724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c66702-feb4-457a-9b15-7abb9d9455f1 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:48.769535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1217/2458] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:50:48 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:50:48.774025 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36cd45-52fe-489a-a67c-3284e9ee1d22 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:48.787761 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36cd45-52fe-489a-a67c-3284e9ee1d22 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:48.821106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:50:48.850959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73843ff-a179-4547-a953-b2b2e3c934c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:48.851843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73843ff-a179-4547-a953-b2b2e3c934c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:48.853692 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73843ff-a179-4547-a953-b2b2e3c934c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:48.854011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73843ff-a179-4547-a953-b2b2e3c934c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:48.854719 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73843ff-a179-4547-a953-b2b2e3c934c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:48.856661 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73843ff-a179-4547-a953-b2b2e3c934c9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:48.882395 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1242/2459] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:50:48 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:50:48.889367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c66702-feb4-457a-9b15-7abb9d9455f1 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:48.900052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c66702-feb4-457a-9b15-7abb9d9455f1 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:48.904914 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb89d83-acb8-4968-97f9-f017ee6f449b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:48.905620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb89d83-acb8-4968-97f9-f017ee6f449b admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:48.905907 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb89d83-acb8-4968-97f9-f017ee6f449b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:48.906216 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb89d83-acb8-4968-97f9-f017ee6f449b admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:48.907460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb89d83-acb8-4968-97f9-f017ee6f449b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:48.910440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb89d83-acb8-4968-97f9-f017ee6f449b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:48.922921 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1218/2460] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:50:48.926986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73843ff-a179-4547-a953-b2b2e3c934c9 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:48.937925 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73843ff-a179-4547-a953-b2b2e3c934c9 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:48.942663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cc6e2-b840-4c2d-a744-8e18a2f4dd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:48.943341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cc6e2-b840-4c2d-a744-8e18a2f4dd46 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:48.943622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cc6e2-b840-4c2d-a744-8e18a2f4dd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:48.943896 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cc6e2-b840-4c2d-a744-8e18a2f4dd46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:48.944629 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cc6e2-b840-4c2d-a744-8e18a2f4dd46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:48.946316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cc6e2-b840-4c2d-a744-8e18a2f4dd46 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:48.956875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1243/2461] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:50:48.961370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb89d83-acb8-4968-97f9-f017ee6f449b admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:48.971988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb89d83-acb8-4968-97f9-f017ee6f449b admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:48.978064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42579e-c681-4632-937c-b0dfe30bc945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:48.979418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42579e-c681-4632-937c-b0dfe30bc945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:48.979703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42579e-c681-4632-937c-b0dfe30bc945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:48.979982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42579e-c681-4632-937c-b0dfe30bc945 admin admin] PATH_INFO: `/v3/projects/c8ffe004b4e44ee6b35079502fd74a9d/users/1e78d34d4721485bbc0b0fc6a5d67e7a/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:48.982945 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:50:49.003376 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42579e-c681-4632-937c-b0dfe30bc945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ffe004b4e44ee6b35079502fd74a9d, user_id=1e78d34d4721485bbc0b0fc6a5d67e7a, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:49.005266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42579e-c681-4632-937c-b0dfe30bc945 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:49.019997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1219/2462] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:50:48 2022] PUT /identity/v3/projects/c8ffe004b4e44ee6b35079502fd74a9d/users/1e78d34d4721485bbc0b0fc6a5d67e7a/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:50:49.032353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523b4da-64bb-4c77-aac0-1679ada0bd35 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:49.032865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523b4da-64bb-4c77-aac0-1679ada0bd35 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:49.032966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523b4da-64bb-4c77-aac0-1679ada0bd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:49.064171 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a523b4da-64bb-4c77-aac0-1679ada0bd35 None None] MFA Rules not processed for user `1e78d34d4721485bbc0b0fc6a5d67e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:50:49.065421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a523b4da-64bb-4c77-aac0-1679ada0bd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:49.135739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1244/2463] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:50:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:50:49.147031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42579e-c681-4632-937c-b0dfe30bc945 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:49.148747 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-da42579e-c681-4632-937c-b0dfe30bc945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:49.231731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42579e-c681-4632-937c-b0dfe30bc945 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:49.237885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cbfb4-0b12-44b7-bdf1-4b9f180295e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:49.238596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cbfb4-0b12-44b7-bdf1-4b9f180295e8 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:49.238887 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cbfb4-0b12-44b7-bdf1-4b9f180295e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:49.239170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cbfb4-0b12-44b7-bdf1-4b9f180295e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:49.248602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354cbfb4-0b12-44b7-bdf1-4b9f180295e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:49.250265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354cbfb4-0b12-44b7-bdf1-4b9f180295e8 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:49.259732 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1220/2464] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:50:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:50:57.190258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523b4da-64bb-4c77-aac0-1679ada0bd35 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:57.191658 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a523b4da-64bb-4c77-aac0-1679ada0bd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:57.256364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523b4da-64bb-4c77-aac0-1679ada0bd35 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:57.261932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f463f-782e-4ff2-bd1a-a99398e09cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:57.262590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f463f-782e-4ff2-bd1a-a99398e09cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:57.262868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f463f-782e-4ff2-bd1a-a99398e09cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:57.263125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f463f-782e-4ff2-bd1a-a99398e09cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:57.265076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5b3f463f-782e-4ff2-bd1a-a99398e09cd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:57.328209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:50:57.334698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f463f-782e-4ff2-bd1a-a99398e09cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:57.337689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f463f-782e-4ff2-bd1a-a99398e09cd0 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:57.350322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1245/2465] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:50:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:50:58.780115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cbfb4-0b12-44b7-bdf1-4b9f180295e8 service nova] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:58.791893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cbfb4-0b12-44b7-bdf1-4b9f180295e8 service nova] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:58.800237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:58.800237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:58.800237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:58.800237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] PATH_INFO: `/v3/users/1e78d34d4721485bbc0b0fc6a5d67e7a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:58.800800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e78d34d4721485bbc0b0fc6a5d67e7a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:58.802820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:58.856470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e78d34d4721485bbc0b0fc6a5d67e7a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:58.857460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e78d34d4721485bbc0b0fc6a5d67e7a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:58.858118 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e78d34d4721485bbc0b0fc6a5d67e7a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:58.865895 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e78d34d4721485bbc0b0fc6a5d67e7a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:58.876578 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e78d34d4721485bbc0b0fc6a5d67e7a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:58.896238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1221/2466] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:50:58 2022] DELETE /identity/v3/users/1e78d34d4721485bbc0b0fc6a5d67e7a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:50:58.900893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0a321-38aa-456b-a37c-9a7676fdc0ac None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:58.901403 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0a321-38aa-456b-a37c-9a7676fdc0ac None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:58.901500 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0a321-38aa-456b-a37c-9a7676fdc0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:58.920932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-36b0a321-38aa-456b-a37c-9a7676fdc0ac None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:50:58.922281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-36b0a321-38aa-456b-a37c-9a7676fdc0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:58.988814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1246/2467] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:50:59.001202 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:59.002813 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:59.059021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:50:59.078146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027771c5-89b7-4575-b9fb-00801083d08b admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:59.083290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d152398-16bb-478d-8c30-1d116b0c647b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:59.084194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d152398-16bb-478d-8c30-1d116b0c647b service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:59.084546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d152398-16bb-478d-8c30-1d116b0c647b service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:59.084891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d152398-16bb-478d-8c30-1d116b0c647b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:59.096521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d152398-16bb-478d-8c30-1d116b0c647b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:59.098203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d152398-16bb-478d-8c30-1d116b0c647b service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:59.114939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1222/2468] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:50:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:50:59.619638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0a321-38aa-456b-a37c-9a7676fdc0ac None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:59.622400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-36b0a321-38aa-456b-a37c-9a7676fdc0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:59.649091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0a321-38aa-456b-a37c-9a7676fdc0ac None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:59.656443 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef6e44-fbab-4161-adac-e05ce2ab5c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:59.657437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef6e44-fbab-4161-adac-e05ce2ab5c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:59.657831 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef6e44-fbab-4161-adac-e05ce2ab5c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:59.658115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef6e44-fbab-4161-adac-e05ce2ab5c06 admin admin] PATH_INFO: `/v3/projects/c8ffe004b4e44ee6b35079502fd74a9d` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:59.659568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef6e44-fbab-4161-adac-e05ce2ab5c06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ffe004b4e44ee6b35079502fd74a9d)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:59.662460 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef6e44-fbab-4161-adac-e05ce2ab5c06 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:59.727486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbef6e44-fbab-4161-adac-e05ce2ab5c06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ffe004b4e44ee6b35079502fd74a9d', 'request_id': 'req-cbef6e44-fbab-4161-adac-e05ce2ab5c06'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:59.735369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbef6e44-fbab-4161-adac-e05ce2ab5c06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ffe004b4e44ee6b35079502fd74a9d', 'request_id': 'req-cbef6e44-fbab-4161-adac-e05ce2ab5c06'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:59.744214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbef6e44-fbab-4161-adac-e05ce2ab5c06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ffe004b4e44ee6b35079502fd74a9d', 'request_id': 'req-cbef6e44-fbab-4161-adac-e05ce2ab5c06'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:59.761973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbef6e44-fbab-4161-adac-e05ce2ab5c06 admin admin] The token cache is being invalidate because project c8ffe004b4e44ee6b35079502fd74a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:50:59.762261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbef6e44-fbab-4161-adac-e05ce2ab5c06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:59.762991 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cbef6e44-fbab-4161-adac-e05ce2ab5c06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:50:59.764566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1247/2469] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:50:59 2022] DELETE /identity/v3/projects/c8ffe004b4e44ee6b35079502fd74a9d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:50:59.769649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e088c-d8a2-476a-987f-2e8ce4482161 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:59.769872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e088c-d8a2-476a-987f-2e8ce4482161 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:59.770185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e088c-d8a2-476a-987f-2e8ce4482161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:59.791843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-2f3e088c-d8a2-476a-987f-2e8ce4482161 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:50:59.793060 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2f3e088c-d8a2-476a-987f-2e8ce4482161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:59.867400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1223/2470] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:50:59.875698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf707da-7858-4e10-b1eb-6cda5821851a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:59.875992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf707da-7858-4e10-b1eb-6cda5821851a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:59.876303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf707da-7858-4e10-b1eb-6cda5821851a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:59.898638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-faf707da-7858-4e10-b1eb-6cda5821851a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:50:59.899940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-faf707da-7858-4e10-b1eb-6cda5821851a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:50:59.907992 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1248/2471] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:50:59.912757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e088c-d8a2-476a-987f-2e8ce4482161 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:59.924889 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e088c-d8a2-476a-987f-2e8ce4482161 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:59.933106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e1616-66cd-4ce5-9e99-58a2d82c7fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:59.933800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e1616-66cd-4ce5-9e99-58a2d82c7fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:59.934111 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e1616-66cd-4ce5-9e99-58a2d82c7fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:59.934395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e1616-66cd-4ce5-9e99-58a2d82c7fa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:59.935165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e1616-66cd-4ce5-9e99-58a2d82c7fa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:50:59.935478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e1616-66cd-4ce5-9e99-58a2d82c7fa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:59.937161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e1616-66cd-4ce5-9e99-58a2d82c7fa0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:50:59.950303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1224/2472] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:50:59.954870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf707da-7858-4e10-b1eb-6cda5821851a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:50:59.972652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf707da-7858-4e10-b1eb-6cda5821851a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:50:59.982512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70b620-57b5-44db-b04b-38c8b347c06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:50:59.983899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70b620-57b5-44db-b04b-38c8b347c06f admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:50:59.984421 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70b620-57b5-44db-b04b-38c8b347c06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:50:59.985132 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70b620-57b5-44db-b04b-38c8b347c06f admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:50:59.986033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70b620-57b5-44db-b04b-38c8b347c06f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:50:59.988355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70b620-57b5-44db-b04b-38c8b347c06f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:00.022208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1249/2473] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:50:59 2022] POST /identity/v3/projects => generated 405 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:00.026880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e1616-66cd-4ce5-9e99-58a2d82c7fa0 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:00.038820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e1616-66cd-4ce5-9e99-58a2d82c7fa0 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:00.122609 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b7d91-d716-45f4-bc6f-cc677d6b93be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:00.123570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b7d91-d716-45f4-bc6f-cc677d6b93be admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:00.124062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b7d91-d716-45f4-bc6f-cc677d6b93be admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:00.124343 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b7d91-d716-45f4-bc6f-cc677d6b93be admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:00.125098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b7d91-d716-45f4-bc6f-cc677d6b93be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:00.127093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b7d91-d716-45f4-bc6f-cc677d6b93be admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:00.150276 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1225/2474] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:00 2022] POST /identity/v3/users => generated 303 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:00.154490 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70b620-57b5-44db-b04b-38c8b347c06f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:00.167820 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70b620-57b5-44db-b04b-38c8b347c06f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:00.172942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583b2a2-5315-4474-a672-5665964d81fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:00.173697 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583b2a2-5315-4474-a672-5665964d81fd admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:00.174117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583b2a2-5315-4474-a672-5665964d81fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:00.174459 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583b2a2-5315-4474-a672-5665964d81fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:00.175297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583b2a2-5315-4474-a672-5665964d81fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:00.177011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583b2a2-5315-4474-a672-5665964d81fd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:00.189775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1250/2475] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:00.193796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b7d91-d716-45f4-bc6f-cc677d6b93be admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:00.204668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b7d91-d716-45f4-bc6f-cc677d6b93be admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:00.210210 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84758776-23ad-4411-9f39-4debb9b3489e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:00.210864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84758776-23ad-4411-9f39-4debb9b3489e admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:00.211140 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84758776-23ad-4411-9f39-4debb9b3489e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:00.211406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84758776-23ad-4411-9f39-4debb9b3489e admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:00.212224 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84758776-23ad-4411-9f39-4debb9b3489e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:00.214077 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84758776-23ad-4411-9f39-4debb9b3489e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:00.226328 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1226/2476] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:00.231541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583b2a2-5315-4474-a672-5665964d81fd admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:00.242746 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583b2a2-5315-4474-a672-5665964d81fd admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:00.249411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861823b-79b2-4b49-9706-0f3c8c6419ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:00.250389 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861823b-79b2-4b49-9706-0f3c8c6419ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:00.250666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861823b-79b2-4b49-9706-0f3c8c6419ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:00.250949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861823b-79b2-4b49-9706-0f3c8c6419ef admin admin] PATH_INFO: `/v3/projects/42d22c665af0480fa75cd0fe952448d2/users/48ba610ef3624a3480221a56518d16fa/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:00.277133 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861823b-79b2-4b49-9706-0f3c8c6419ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d22c665af0480fa75cd0fe952448d2, user_id=48ba610ef3624a3480221a56518d16fa, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:00.279905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861823b-79b2-4b49-9706-0f3c8c6419ef admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:00.301942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1251/2477] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:00 2022] PUT /identity/v3/projects/42d22c665af0480fa75cd0fe952448d2/users/48ba610ef3624a3480221a56518d16fa/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:51:00.303836 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2837f-340a-4e17-803a-3b5e1fab35bc None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:00.304227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2837f-340a-4e17-803a-3b5e1fab35bc None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:00.304598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2837f-340a-4e17-803a-3b5e1fab35bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:00.326211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-65b2837f-340a-4e17-803a-3b5e1fab35bc None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:00.327630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-65b2837f-340a-4e17-803a-3b5e1fab35bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:00.392283 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:00.425339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1227/2478] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:00.437839 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861823b-79b2-4b49-9706-0f3c8c6419ef admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:00.439834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4861823b-79b2-4b49-9706-0f3c8c6419ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:00.464313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:00.521277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861823b-79b2-4b49-9706-0f3c8c6419ef admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:00.530367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde8436-cf78-4cd0-b3d2-18b35abfde77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:00.531399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde8436-cf78-4cd0-b3d2-18b35abfde77 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:00.531990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde8436-cf78-4cd0-b3d2-18b35abfde77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:00.532465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde8436-cf78-4cd0-b3d2-18b35abfde77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:00.544318 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde8436-cf78-4cd0-b3d2-18b35abfde77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:00.545780 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde8436-cf78-4cd0-b3d2-18b35abfde77 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:00.559039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1252/2479] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:00.947697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2837f-340a-4e17-803a-3b5e1fab35bc None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:00.949194 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-65b2837f-340a-4e17-803a-3b5e1fab35bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:00.962800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2837f-340a-4e17-803a-3b5e1fab35bc None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:00.968703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e2ee2-c3b3-46ba-96e9-02256bece825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:00.969975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e2ee2-c3b3-46ba-96e9-02256bece825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:00.970675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e2ee2-c3b3-46ba-96e9-02256bece825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:00.970943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e2ee2-c3b3-46ba-96e9-02256bece825 admin admin] PATH_INFO: `/v3/users/1bc5799ac34349518854829ec2c259f9` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:00.972324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e2ee2-c3b3-46ba-96e9-02256bece825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc5799ac34349518854829ec2c259f9)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:00.974729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e2ee2-c3b3-46ba-96e9-02256bece825 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:01.052795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-382e2ee2-c3b3-46ba-96e9-02256bece825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc5799ac34349518854829ec2c259f9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:01.053510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-382e2ee2-c3b3-46ba-96e9-02256bece825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc5799ac34349518854829ec2c259f9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:01.054154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-382e2ee2-c3b3-46ba-96e9-02256bece825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc5799ac34349518854829ec2c259f9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:01.064951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-382e2ee2-c3b3-46ba-96e9-02256bece825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc5799ac34349518854829ec2c259f9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:01.075322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-382e2ee2-c3b3-46ba-96e9-02256bece825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc5799ac34349518854829ec2c259f9'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:01.108442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1228/2480] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:00 2022] DELETE /identity/v3/users/1bc5799ac34349518854829ec2c259f9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:01.318668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde8436-cf78-4cd0-b3d2-18b35abfde77 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:01.337166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde8436-cf78-4cd0-b3d2-18b35abfde77 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:01.404383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b72b87-eea8-471f-a356-f6eae200a36a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.404675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b72b87-eea8-471f-a356-f6eae200a36a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.404957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b72b87-eea8-471f-a356-f6eae200a36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.427396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659bb8e-9c56-4fa2-9706-0c013058196b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:01.428706 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659bb8e-9c56-4fa2-9706-0c013058196b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.429244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659bb8e-9c56-4fa2-9706-0c013058196b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.429588 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659bb8e-9c56-4fa2-9706-0c013058196b admin admin] PATH_INFO: `/v3/projects/df3ed3c23dde49b382ce9016e421e4de` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.431686 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659bb8e-9c56-4fa2-9706-0c013058196b admin admin] RBAC: Authorizing `identity:delete_project(project_id=df3ed3c23dde49b382ce9016e421e4de)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:01.433750 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659bb8e-9c56-4fa2-9706-0c013058196b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:01.435234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-83b72b87-eea8-471f-a356-f6eae200a36a None None] MFA Rules not processed for user `48ba610ef3624a3480221a56518d16fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:01.436538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-83b72b87-eea8-471f-a356-f6eae200a36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:01.496221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5659bb8e-9c56-4fa2-9706-0c013058196b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df3ed3c23dde49b382ce9016e421e4de', 'request_id': 'req-5659bb8e-9c56-4fa2-9706-0c013058196b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:01.504966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5659bb8e-9c56-4fa2-9706-0c013058196b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df3ed3c23dde49b382ce9016e421e4de', 'request_id': 'req-5659bb8e-9c56-4fa2-9706-0c013058196b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:01.517595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5659bb8e-9c56-4fa2-9706-0c013058196b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df3ed3c23dde49b382ce9016e421e4de', 'request_id': 'req-5659bb8e-9c56-4fa2-9706-0c013058196b'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:01.519070 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1229/2481] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:01 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:51:01.523599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b72b87-eea8-471f-a356-f6eae200a36a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:01.526219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-83b72b87-eea8-471f-a356-f6eae200a36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:01.537495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5659bb8e-9c56-4fa2-9706-0c013058196b admin admin] The token cache is being invalidate because project df3ed3c23dde49b382ce9016e421e4de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:01.538406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5659bb8e-9c56-4fa2-9706-0c013058196b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:01.539553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5659bb8e-9c56-4fa2-9706-0c013058196b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:01.540940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1253/2482] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:01 2022] DELETE /identity/v3/projects/df3ed3c23dde49b382ce9016e421e4de => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:01.547318 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af666e60-2df3-4ebf-a352-4b94e20be4b2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.547767 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af666e60-2df3-4ebf-a352-4b94e20be4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.548638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af666e60-2df3-4ebf-a352-4b94e20be4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.550875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b72b87-eea8-471f-a356-f6eae200a36a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:01.574159 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-af666e60-2df3-4ebf-a352-4b94e20be4b2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:01.575794 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-af666e60-2df3-4ebf-a352-4b94e20be4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:01.612111 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:01.627444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de6ef1-500b-4d6e-b474-a3909b877f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:01.628137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de6ef1-500b-4d6e-b474-a3909b877f7b admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.628418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de6ef1-500b-4d6e-b474-a3909b877f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.628692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de6ef1-500b-4d6e-b474-a3909b877f7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.629387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de6ef1-500b-4d6e-b474-a3909b877f7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:01.631535 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de6ef1-500b-4d6e-b474-a3909b877f7b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:01.635052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:01.658861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1254/2483] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:51:01.664951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1230/2484] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:01 2022] POST /identity/v3/projects => generated 403 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:01.666105 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af666e60-2df3-4ebf-a352-4b94e20be4b2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:01.667323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b0654-3b06-4914-ba17-17d8f5117fa3 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.667638 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b0654-3b06-4914-ba17-17d8f5117fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.667843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-af666e60-2df3-4ebf-a352-4b94e20be4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:01.668005 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b0654-3b06-4914-ba17-17d8f5117fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.680052 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af666e60-2df3-4ebf-a352-4b94e20be4b2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:01.688135 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-d96b0654-3b06-4914-ba17-17d8f5117fa3 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:01.689386 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d96b0654-3b06-4914-ba17-17d8f5117fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:01.739462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e40cd7-eab8-4ac5-bcb8-f298a0d23377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:01.740177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e40cd7-eab8-4ac5-bcb8-f298a0d23377 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.740456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e40cd7-eab8-4ac5-bcb8-f298a0d23377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.740729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e40cd7-eab8-4ac5-bcb8-f298a0d23377 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.741479 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e40cd7-eab8-4ac5-bcb8-f298a0d23377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:01.743572 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e40cd7-eab8-4ac5-bcb8-f298a0d23377 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:01.757815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1231/2485] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:51:01.763757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b0654-3b06-4914-ba17-17d8f5117fa3 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:01.770758 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1255/2486] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:01 2022] POST /identity/v3/users => generated 308 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:01.771819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e40cd7-eab8-4ac5-bcb8-f298a0d23377 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:01.773482 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b0654-3b06-4914-ba17-17d8f5117fa3 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:01.779048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b7d8b-172e-4524-839e-36092997db6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:01.779737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b7d8b-172e-4524-839e-36092997db6a admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.780025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b7d8b-172e-4524-839e-36092997db6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.780306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b7d8b-172e-4524-839e-36092997db6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.781058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b7d8b-172e-4524-839e-36092997db6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:51:01.781369 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b7d8b-172e-4524-839e-36092997db6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:01.783057 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b7d8b-172e-4524-839e-36092997db6a admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:01.785805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e40cd7-eab8-4ac5-bcb8-f298a0d23377 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:01.790789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd1977-9be1-4033-9510-bb12a7d3d546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:01.791473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd1977-9be1-4033-9510-bb12a7d3d546 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.791760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd1977-9be1-4033-9510-bb12a7d3d546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.792036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd1977-9be1-4033-9510-bb12a7d3d546 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.792929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd1977-9be1-4033-9510-bb12a7d3d546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:01.794586 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd1977-9be1-4033-9510-bb12a7d3d546 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:01.796992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1232/2487] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:51:01.801401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b7d8b-172e-4524-839e-36092997db6a admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:01.810518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1256/2488] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:51:01.811568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd1977-9be1-4033-9510-bb12a7d3d546 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:01.815006 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b7d8b-172e-4524-839e-36092997db6a admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:01.819732 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dee11f-e7a4-45cd-a49d-7c5cac64c371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:01.820392 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dee11f-e7a4-45cd-a49d-7c5cac64c371 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.820675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dee11f-e7a4-45cd-a49d-7c5cac64c371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.820952 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dee11f-e7a4-45cd-a49d-7c5cac64c371 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.822326 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dee11f-e7a4-45cd-a49d-7c5cac64c371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:01.822462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd1977-9be1-4033-9510-bb12a7d3d546 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:01.824296 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dee11f-e7a4-45cd-a49d-7c5cac64c371 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:01.827298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc4da5-a131-4531-ac52-2c0a6f0cdfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:01.828163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc4da5-a131-4531-ac52-2c0a6f0cdfaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.828448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc4da5-a131-4531-ac52-2c0a6f0cdfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.828755 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc4da5-a131-4531-ac52-2c0a6f0cdfaf admin admin] PATH_INFO: `/v3/projects/efdf9399556b4babbe4e5098a9d8d310/users/e8b3745b6fe24fd2ad0ad4fab751f259/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.846717 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc4da5-a131-4531-ac52-2c0a6f0cdfaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=efdf9399556b4babbe4e5098a9d8d310, user_id=e8b3745b6fe24fd2ad0ad4fab751f259, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:01.849342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc4da5-a131-4531-ac52-2c0a6f0cdfaf admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:01.857458 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1233/2489] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:01 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:01.862260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dee11f-e7a4-45cd-a49d-7c5cac64c371 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:01.864663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1257/2490] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:01 2022] PUT /identity/v3/projects/efdf9399556b4babbe4e5098a9d8d310/users/e8b3745b6fe24fd2ad0ad4fab751f259/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:51:01.868367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc4da5-a131-4531-ac52-2c0a6f0cdfaf admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:01.934140 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc4da5-a131-4531-ac52-2c0a6f0cdfaf admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:01.935112 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dee11f-e7a4-45cd-a49d-7c5cac64c371 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:01.940970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c43dd-c12c-436f-a9ae-c929586ce8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:01.941645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c43dd-c12c-436f-a9ae-c929586ce8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.941936 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c43dd-c12c-436f-a9ae-c929586ce8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.942212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c43dd-c12c-436f-a9ae-c929586ce8e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.942488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6be6a4-cee8-44e8-af60-07ca323701c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:01.942943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c43dd-c12c-436f-a9ae-c929586ce8e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:01.943169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6be6a4-cee8-44e8-af60-07ca323701c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.943454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6be6a4-cee8-44e8-af60-07ca323701c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.943727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6be6a4-cee8-44e8-af60-07ca323701c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.944422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6be6a4-cee8-44e8-af60-07ca323701c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:01.944530 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c43dd-c12c-436f-a9ae-c929586ce8e2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:01.946951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6be6a4-cee8-44e8-af60-07ca323701c9 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:01.955781 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1258/2491] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:51:01.960260 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c43dd-c12c-436f-a9ae-c929586ce8e2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:01.971451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c43dd-c12c-436f-a9ae-c929586ce8e2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:01.972289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1234/2492] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:01 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:01.976531 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6be6a4-cee8-44e8-af60-07ca323701c9 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:01.979309 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb549c71-1305-449d-b7ba-e784316f2bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:01.980236 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb549c71-1305-449d-b7ba-e784316f2bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:01.980520 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb549c71-1305-449d-b7ba-e784316f2bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:01.980801 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb549c71-1305-449d-b7ba-e784316f2bf6 admin admin] PATH_INFO: `/v3/projects/efdf9399556b4babbe4e5098a9d8d310/users/e8b3745b6fe24fd2ad0ad4fab751f259/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:01.983327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb549c71-1305-449d-b7ba-e784316f2bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efdf9399556b4babbe4e5098a9d8d310, user_id=e8b3745b6fe24fd2ad0ad4fab751f259, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:01.986087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb549c71-1305-449d-b7ba-e784316f2bf6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:01.990876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6be6a4-cee8-44e8-af60-07ca323701c9 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:01.998373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1259/2493] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:01 2022] PUT /identity/v3/projects/efdf9399556b4babbe4e5098a9d8d310/users/e8b3745b6fe24fd2ad0ad4fab751f259/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:51:02.027753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:02.059711 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346eabb-8c16-416e-9e75-078679110296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:02.060532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346eabb-8c16-416e-9e75-078679110296 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:02.060824 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346eabb-8c16-416e-9e75-078679110296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:02.061122 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346eabb-8c16-416e-9e75-078679110296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:02.061851 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346eabb-8c16-416e-9e75-078679110296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:02.063552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346eabb-8c16-416e-9e75-078679110296 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:02.074965 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1235/2494] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:51:02.079263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb549c71-1305-449d-b7ba-e784316f2bf6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:02.093528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb549c71-1305-449d-b7ba-e784316f2bf6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:02.098713 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc62b5-ee5a-4cc3-802f-29213fc33057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:02.099394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc62b5-ee5a-4cc3-802f-29213fc33057 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:02.099674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc62b5-ee5a-4cc3-802f-29213fc33057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:02.099978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc62b5-ee5a-4cc3-802f-29213fc33057 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:02.100738 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc62b5-ee5a-4cc3-802f-29213fc33057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:02.104508 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc62b5-ee5a-4cc3-802f-29213fc33057 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:02.113662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1260/2495] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:02.118607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346eabb-8c16-416e-9e75-078679110296 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:02.128686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346eabb-8c16-416e-9e75-078679110296 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:02.134433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06be9d-0342-4662-bf75-f5b5d4fc3090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:02.135516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06be9d-0342-4662-bf75-f5b5d4fc3090 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:02.135803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06be9d-0342-4662-bf75-f5b5d4fc3090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:02.136085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06be9d-0342-4662-bf75-f5b5d4fc3090 admin admin] PATH_INFO: `/v3/projects/a89ad1e525364d3787f0643e8104a532/users/2f0dfc223e54438788758f115fb02eab/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:02.153019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06be9d-0342-4662-bf75-f5b5d4fc3090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89ad1e525364d3787f0643e8104a532, user_id=2f0dfc223e54438788758f115fb02eab, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:02.156213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06be9d-0342-4662-bf75-f5b5d4fc3090 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:02.174795 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1236/2496] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:02 2022] PUT /identity/v3/projects/a89ad1e525364d3787f0643e8104a532/users/2f0dfc223e54438788758f115fb02eab/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:51:02.187702 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b020372-5b3f-4600-b13b-d89288f75757 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:02.188068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b020372-5b3f-4600-b13b-d89288f75757 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:02.188358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b020372-5b3f-4600-b13b-d89288f75757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:02.216863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:02.221358 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1b020372-5b3f-4600-b13b-d89288f75757 None None] MFA Rules not processed for user `2f0dfc223e54438788758f115fb02eab`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:02.223610 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1b020372-5b3f-4600-b13b-d89288f75757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:02.296268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1261/2497] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:02.308808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06be9d-0342-4662-bf75-f5b5d4fc3090 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:02.310684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-1f06be9d-0342-4662-bf75-f5b5d4fc3090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:02.403415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06be9d-0342-4662-bf75-f5b5d4fc3090 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:02.409284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09f9a1-f9a5-4591-bc61-570666e189fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:02.409997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09f9a1-f9a5-4591-bc61-570666e189fb service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:02.410286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09f9a1-f9a5-4591-bc61-570666e189fb service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:02.410565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09f9a1-f9a5-4591-bc61-570666e189fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:02.419966 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09f9a1-f9a5-4591-bc61-570666e189fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:02.421608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09f9a1-f9a5-4591-bc61-570666e189fb service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:02.432510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1237/2498] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:02.985762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888df00d-63fc-43c2-aa89-e553323a879a None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:02.986366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888df00d-63fc-43c2-aa89-e553323a879a None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:02.986366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888df00d-63fc-43c2-aa89-e553323a879a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:03.019821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-888df00d-63fc-43c2-aa89-e553323a879a None None] MFA Rules not processed for user `e8b3745b6fe24fd2ad0ad4fab751f259`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:03.021342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-888df00d-63fc-43c2-aa89-e553323a879a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:03.097044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1262/2499] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:04.218177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09f9a1-f9a5-4591-bc61-570666e189fb service nova] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:04.286932 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09f9a1-f9a5-4591-bc61-570666e189fb service nova] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:04.292724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:04.293775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:04.294236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:04.294517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] PATH_INFO: `/v3/users/48ba610ef3624a3480221a56518d16fa` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:04.295892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ba610ef3624a3480221a56518d16fa)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:04.298640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:04.335937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:04.352223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ba610ef3624a3480221a56518d16fa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.352975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ba610ef3624a3480221a56518d16fa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.353582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ba610ef3624a3480221a56518d16fa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.361554 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ba610ef3624a3480221a56518d16fa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.372632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ba610ef3624a3480221a56518d16fa'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.395964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1238/2500] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:04 2022] DELETE /identity/v3/users/48ba610ef3624a3480221a56518d16fa => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:04.585512 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888df00d-63fc-43c2-aa89-e553323a879a None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:04.596880 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888df00d-63fc-43c2-aa89-e553323a879a None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:04.654533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:04.655392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:04.655670 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:04.655941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] PATH_INFO: `/v3/projects/42d22c665af0480fa75cd0fe952448d2` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:04.657153 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d22c665af0480fa75cd0fe952448d2)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:04.659113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:04.695533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:04.712029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d22c665af0480fa75cd0fe952448d2', 'request_id': 'req-d7507338-ce74-400f-95da-6149f3e91153'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.721984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d22c665af0480fa75cd0fe952448d2', 'request_id': 'req-d7507338-ce74-400f-95da-6149f3e91153'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.729333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d22c665af0480fa75cd0fe952448d2', 'request_id': 'req-d7507338-ce74-400f-95da-6149f3e91153'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.743601 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] The token cache is being invalidate because project 42d22c665af0480fa75cd0fe952448d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:04.743912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.744614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.746203 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1263/2501] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:04 2022] DELETE /identity/v3/projects/42d22c665af0480fa75cd0fe952448d2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:04.750212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:04.751608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:04.764837 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364d57-7752-4564-b2fa-a270a527daaa admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:04.824225 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:04.825622 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:04.826219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:04.826701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] PATH_INFO: `/v3/users/e8b3745b6fe24fd2ad0ad4fab751f259` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:04.828545 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8b3745b6fe24fd2ad0ad4fab751f259)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:04.831762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:04.881445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8b3745b6fe24fd2ad0ad4fab751f259'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.882034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8b3745b6fe24fd2ad0ad4fab751f259'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.882904 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8b3745b6fe24fd2ad0ad4fab751f259'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.889810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8b3745b6fe24fd2ad0ad4fab751f259'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.897292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8b3745b6fe24fd2ad0ad4fab751f259'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:04.914507 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1239/2502] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:04 2022] DELETE /identity/v3/users/e8b3745b6fe24fd2ad0ad4fab751f259 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:05.097292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:05.098913 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:05.113442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7507338-ce74-400f-95da-6149f3e91153 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:05.179710 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb31277-a3e1-4452-8c30-1bacfbf2eace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:05.180615 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb31277-a3e1-4452-8c30-1bacfbf2eace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.180895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb31277-a3e1-4452-8c30-1bacfbf2eace admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.181170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb31277-a3e1-4452-8c30-1bacfbf2eace admin admin] PATH_INFO: `/v3/projects/efdf9399556b4babbe4e5098a9d8d310` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.182496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb31277-a3e1-4452-8c30-1bacfbf2eace admin admin] RBAC: Authorizing `identity:delete_project(project_id=efdf9399556b4babbe4e5098a9d8d310)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:05.184662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb31277-a3e1-4452-8c30-1bacfbf2eace admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:05.215697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:05.215697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:05.230109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792b4c6-7d1c-465b-bd6b-bddc9f248357 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:05.246430 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4eb31277-a3e1-4452-8c30-1bacfbf2eace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efdf9399556b4babbe4e5098a9d8d310', 'request_id': 'req-4eb31277-a3e1-4452-8c30-1bacfbf2eace'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:05.256416 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4eb31277-a3e1-4452-8c30-1bacfbf2eace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efdf9399556b4babbe4e5098a9d8d310', 'request_id': 'req-4eb31277-a3e1-4452-8c30-1bacfbf2eace'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:05.258401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:05.266255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4eb31277-a3e1-4452-8c30-1bacfbf2eace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efdf9399556b4babbe4e5098a9d8d310', 'request_id': 'req-4eb31277-a3e1-4452-8c30-1bacfbf2eace'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:05.280308 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4eb31277-a3e1-4452-8c30-1bacfbf2eace admin admin] The token cache is being invalidate because project efdf9399556b4babbe4e5098a9d8d310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:05.280597 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4eb31277-a3e1-4452-8c30-1bacfbf2eace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:05.281263 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-4eb31277-a3e1-4452-8c30-1bacfbf2eace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:05.283180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1264/2503] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:05 2022] DELETE /identity/v3/projects/efdf9399556b4babbe4e5098a9d8d310 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:51:05.288518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac995d0-270d-483d-9f3d-ee27c38b285b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.288806 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac995d0-270d-483d-9f3d-ee27c38b285b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.289088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac995d0-270d-483d-9f3d-ee27c38b285b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.299757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a3017-6985-489d-931f-68f5440a8df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:05.300739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a3017-6985-489d-931f-68f5440a8df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.301020 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a3017-6985-489d-931f-68f5440a8df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.301325 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a3017-6985-489d-931f-68f5440a8df7 admin admin] PATH_INFO: `/v3/users/2f0dfc223e54438788758f115fb02eab` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.302553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a3017-6985-489d-931f-68f5440a8df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f0dfc223e54438788758f115fb02eab)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:05.304500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a3017-6985-489d-931f-68f5440a8df7 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:05.313032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-8ac995d0-270d-483d-9f3d-ee27c38b285b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:05.314353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8ac995d0-270d-483d-9f3d-ee27c38b285b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:05.326014 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1265/2504] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:05.334198 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765ec9c-0a9a-4e48-915b-76f6247cd4aa None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.334499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765ec9c-0a9a-4e48-915b-76f6247cd4aa None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.334965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765ec9c-0a9a-4e48-915b-76f6247cd4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.357015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8b1a3017-6985-489d-931f-68f5440a8df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f0dfc223e54438788758f115fb02eab'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:05.358030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8b1a3017-6985-489d-931f-68f5440a8df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f0dfc223e54438788758f115fb02eab'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:05.358699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8b1a3017-6985-489d-931f-68f5440a8df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f0dfc223e54438788758f115fb02eab'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:05.359326 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-1765ec9c-0a9a-4e48-915b-76f6247cd4aa None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:05.360543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1765ec9c-0a9a-4e48-915b-76f6247cd4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:05.367350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8b1a3017-6985-489d-931f-68f5440a8df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f0dfc223e54438788758f115fb02eab'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:05.369850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1266/2505] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:51:05.374120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765ec9c-0a9a-4e48-915b-76f6247cd4aa None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:05.375878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8b1a3017-6985-489d-931f-68f5440a8df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f0dfc223e54438788758f115fb02eab'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:05.384155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765ec9c-0a9a-4e48-915b-76f6247cd4aa None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:05.389733 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a93aa4-2bd7-4879-bbd2-c45efeee37d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:05.390461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a93aa4-2bd7-4879-bbd2-c45efeee37d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.390747 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a93aa4-2bd7-4879-bbd2-c45efeee37d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.391021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a93aa4-2bd7-4879-bbd2-c45efeee37d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.391769 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a93aa4-2bd7-4879-bbd2-c45efeee37d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:51:05.392078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a93aa4-2bd7-4879-bbd2-c45efeee37d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:05.393685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a93aa4-2bd7-4879-bbd2-c45efeee37d7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:05.396647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1240/2506] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:05 2022] DELETE /identity/v3/users/2f0dfc223e54438788758f115fb02eab => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:05.401717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b755fa9-1fcb-4024-add2-435cbb0d8919 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.402035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b755fa9-1fcb-4024-add2-435cbb0d8919 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.402320 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b755fa9-1fcb-4024-add2-435cbb0d8919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.406124 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1267/2507] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:51:05.410537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a93aa4-2bd7-4879-bbd2-c45efeee37d7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:05.421371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a93aa4-2bd7-4879-bbd2-c45efeee37d7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:05.422818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-4b755fa9-1fcb-4024-add2-435cbb0d8919 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:05.424033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4b755fa9-1fcb-4024-add2-435cbb0d8919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:05.440759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:05.481802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b333883-dd3d-4b56-88c0-9f05943062a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:05.482760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b333883-dd3d-4b56-88c0-9f05943062a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.483040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b333883-dd3d-4b56-88c0-9f05943062a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.483313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b333883-dd3d-4b56-88c0-9f05943062a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.484003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b333883-dd3d-4b56-88c0-9f05943062a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:05.486050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b333883-dd3d-4b56-88c0-9f05943062a8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:05.488063 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1241/2508] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:51:05.498708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b755fa9-1fcb-4024-add2-435cbb0d8919 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:05.500272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4b755fa9-1fcb-4024-add2-435cbb0d8919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:05.519836 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1268/2509] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:05 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:05.524327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b333883-dd3d-4b56-88c0-9f05943062a8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:05.560423 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:05.566454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b755fa9-1fcb-4024-add2-435cbb0d8919 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:05.593471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b333883-dd3d-4b56-88c0-9f05943062a8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:05.600439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b511e3-1b7c-418a-b6aa-93a636f88b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:05.600439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b511e3-1b7c-418a-b6aa-93a636f88b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.600439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b511e3-1b7c-418a-b6aa-93a636f88b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.600439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b511e3-1b7c-418a-b6aa-93a636f88b53 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.602378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b511e3-1b7c-418a-b6aa-93a636f88b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:05.602494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b511e3-1b7c-418a-b6aa-93a636f88b53 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:05.627010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1269/2510] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:05 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:05.631570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b511e3-1b7c-418a-b6aa-93a636f88b53 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:05.631799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b121296-034f-485e-8ddb-925f46f6a744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:05.632422 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b121296-034f-485e-8ddb-925f46f6a744 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.632703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b121296-034f-485e-8ddb-925f46f6a744 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.632979 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b121296-034f-485e-8ddb-925f46f6a744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.642473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b511e3-1b7c-418a-b6aa-93a636f88b53 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:05.643152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b121296-034f-485e-8ddb-925f46f6a744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:05.644596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b121296-034f-485e-8ddb-925f46f6a744 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:05.650212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12482e6-f6dd-496f-82e3-2adab98a9e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:05.650212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12482e6-f6dd-496f-82e3-2adab98a9e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.650212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12482e6-f6dd-496f-82e3-2adab98a9e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.650212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12482e6-f6dd-496f-82e3-2adab98a9e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.650678 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12482e6-f6dd-496f-82e3-2adab98a9e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:05.651676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12482e6-f6dd-496f-82e3-2adab98a9e78 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:05.655652 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1242/2511] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:51:05.662560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1270/2512] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:51:05.666770 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b121296-034f-485e-8ddb-925f46f6a744 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:05.676507 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b121296-034f-485e-8ddb-925f46f6a744 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:05.681613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f48a7-944e-4fbd-a73d-c00fcd93968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:05.682331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f48a7-944e-4fbd-a73d-c00fcd93968a admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.682620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f48a7-944e-4fbd-a73d-c00fcd93968a admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.682897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f48a7-944e-4fbd-a73d-c00fcd93968a admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.683629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f48a7-944e-4fbd-a73d-c00fcd93968a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:05.685257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f48a7-944e-4fbd-a73d-c00fcd93968a admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:05.696677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1243/2513] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:05.701038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12482e6-f6dd-496f-82e3-2adab98a9e78 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:05.712967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12482e6-f6dd-496f-82e3-2adab98a9e78 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:05.718185 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036b0d8-7b41-4958-bdd5-7dce93f9c76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:05.719379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036b0d8-7b41-4958-bdd5-7dce93f9c76f admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.719660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036b0d8-7b41-4958-bdd5-7dce93f9c76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.720145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036b0d8-7b41-4958-bdd5-7dce93f9c76f admin admin] PATH_INFO: `/v3/projects/2e17abf1718a4b5eaa8c31d6fd0c8af8/users/cae13ad7a42644c4be9b792e51b12e09/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.737454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036b0d8-7b41-4958-bdd5-7dce93f9c76f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e17abf1718a4b5eaa8c31d6fd0c8af8, user_id=cae13ad7a42644c4be9b792e51b12e09, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:05.740516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036b0d8-7b41-4958-bdd5-7dce93f9c76f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:05.755123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1271/2514] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:05 2022] PUT /identity/v3/projects/2e17abf1718a4b5eaa8c31d6fd0c8af8/users/cae13ad7a42644c4be9b792e51b12e09/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:05.766137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead515d-eba8-4398-aae7-07f28af8c880 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.766428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead515d-eba8-4398-aae7-07f28af8c880 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.766835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead515d-eba8-4398-aae7-07f28af8c880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.798052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-0ead515d-eba8-4398-aae7-07f28af8c880 None None] MFA Rules not processed for user `cae13ad7a42644c4be9b792e51b12e09`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:05.799333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0ead515d-eba8-4398-aae7-07f28af8c880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:05.875337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1244/2515] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:05.879940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036b0d8-7b41-4958-bdd5-7dce93f9c76f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:05.951380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036b0d8-7b41-4958-bdd5-7dce93f9c76f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:05.953204 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead515d-eba8-4398-aae7-07f28af8c880 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:05.954275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b76394-4c7d-4184-a6aa-1a9ea96c1b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:05.954999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b76394-4c7d-4184-a6aa-1a9ea96c1b1d admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.955223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0ead515d-eba8-4398-aae7-07f28af8c880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:05.955331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b76394-4c7d-4184-a6aa-1a9ea96c1b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.955494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b76394-4c7d-4184-a6aa-1a9ea96c1b1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.956194 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b76394-4c7d-4184-a6aa-1a9ea96c1b1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:05.958142 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b76394-4c7d-4184-a6aa-1a9ea96c1b1d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:05.965222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:05.988112 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead515d-eba8-4398-aae7-07f28af8c880 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:05.994059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30df47-8d8c-4a31-8a35-74419a07e357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:05.995010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30df47-8d8c-4a31-8a35-74419a07e357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:05.995306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30df47-8d8c-4a31-8a35-74419a07e357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:05.995595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30df47-8d8c-4a31-8a35-74419a07e357 admin admin] PATH_INFO: `/v3/projects/a89ad1e525364d3787f0643e8104a532` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:05.996787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30df47-8d8c-4a31-8a35-74419a07e357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89ad1e525364d3787f0643e8104a532)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:05.998539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1272/2516] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:05 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:05.998812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30df47-8d8c-4a31-8a35-74419a07e357 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.002622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b76394-4c7d-4184-a6aa-1a9ea96c1b1d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.054102 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-df30df47-8d8c-4a31-8a35-74419a07e357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89ad1e525364d3787f0643e8104a532', 'request_id': 'req-df30df47-8d8c-4a31-8a35-74419a07e357'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:06.061225 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-df30df47-8d8c-4a31-8a35-74419a07e357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89ad1e525364d3787f0643e8104a532', 'request_id': 'req-df30df47-8d8c-4a31-8a35-74419a07e357'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:06.068063 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-df30df47-8d8c-4a31-8a35-74419a07e357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89ad1e525364d3787f0643e8104a532', 'request_id': 'req-df30df47-8d8c-4a31-8a35-74419a07e357'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:06.074175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b76394-4c7d-4184-a6aa-1a9ea96c1b1d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.081484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-df30df47-8d8c-4a31-8a35-74419a07e357 admin admin] The token cache is being invalidate because project a89ad1e525364d3787f0643e8104a532 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:06.081999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-df30df47-8d8c-4a31-8a35-74419a07e357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:06.082660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-df30df47-8d8c-4a31-8a35-74419a07e357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:06.084520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1245/2517] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:05 2022] DELETE /identity/v3/projects/a89ad1e525364d3787f0643e8104a532 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:06.089812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3c44c-121e-4ffd-b95e-141d95405f6d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.090279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3c44c-121e-4ffd-b95e-141d95405f6d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.090625 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3c44c-121e-4ffd-b95e-141d95405f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.106087 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:06.114528 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f7f3c44c-121e-4ffd-b95e-141d95405f6d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:06.116694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f7f3c44c-121e-4ffd-b95e-141d95405f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:06.137087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa12ed-d5b3-410a-8153-aa5898315e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.137770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa12ed-d5b3-410a-8153-aa5898315e09 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.138087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa12ed-d5b3-410a-8153-aa5898315e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.138374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa12ed-d5b3-410a-8153-aa5898315e09 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.139070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa12ed-d5b3-410a-8153-aa5898315e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.141579 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa12ed-d5b3-410a-8153-aa5898315e09 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.164328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1273/2518] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:06 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:06.171751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa12ed-d5b3-410a-8153-aa5898315e09 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.173532 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-6efa12ed-d5b3-410a-8153-aa5898315e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:06.186269 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1246/2519] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:06.186478 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa12ed-d5b3-410a-8153-aa5898315e09 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.191932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e5255-3731-4cdb-af36-6a0d3cf53a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.192622 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e5255-3731-4cdb-af36-6a0d3cf53a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.192913 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e5255-3731-4cdb-af36-6a0d3cf53a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.193224 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e5255-3731-4cdb-af36-6a0d3cf53a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.193983 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e5255-3731-4cdb-af36-6a0d3cf53a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.194742 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a110e-e204-4a92-8ef1-1fd2fc5520cd None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.195019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a110e-e204-4a92-8ef1-1fd2fc5520cd None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.195258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a110e-e204-4a92-8ef1-1fd2fc5520cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.195637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e5255-3731-4cdb-af36-6a0d3cf53a39 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.207175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1274/2520] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:06.211299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e5255-3731-4cdb-af36-6a0d3cf53a39 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.219236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-af1a110e-e204-4a92-8ef1-1fd2fc5520cd None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:06.220461 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-af1a110e-e204-4a92-8ef1-1fd2fc5520cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:06.221338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e5255-3731-4cdb-af36-6a0d3cf53a39 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.225948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06949e7-1925-4ded-b254-9a8aa25398fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.226905 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06949e7-1925-4ded-b254-9a8aa25398fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.227188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06949e7-1925-4ded-b254-9a8aa25398fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.228020 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06949e7-1925-4ded-b254-9a8aa25398fe admin admin] PATH_INFO: `/v3/projects/6eddc743065649d19400019e9b59f743/users/f56db6ef86894b749bee8d2337991427/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.229647 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1247/2521] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:51:06.234106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a110e-e204-4a92-8ef1-1fd2fc5520cd None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.246088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a110e-e204-4a92-8ef1-1fd2fc5520cd None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.247724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06949e7-1925-4ded-b254-9a8aa25398fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eddc743065649d19400019e9b59f743, user_id=f56db6ef86894b749bee8d2337991427, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.250486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06949e7-1925-4ded-b254-9a8aa25398fe admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.251253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3527eb5-f09e-4794-a606-f3f3a5198ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.251936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3527eb5-f09e-4794-a606-f3f3a5198ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.252216 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3527eb5-f09e-4794-a606-f3f3a5198ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.252491 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3527eb5-f09e-4794-a606-f3f3a5198ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.253226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3527eb5-f09e-4794-a606-f3f3a5198ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:51:06.253533 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3527eb5-f09e-4794-a606-f3f3a5198ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.255186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3527eb5-f09e-4794-a606-f3f3a5198ee3 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.266876 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1275/2522] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:06 2022] PUT /identity/v3/projects/6eddc743065649d19400019e9b59f743/users/f56db6ef86894b749bee8d2337991427/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:51:06.271587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1248/2523] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:51:06.272727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3527eb5-f09e-4794-a606-f3f3a5198ee3 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.272867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06949e7-1925-4ded-b254-9a8aa25398fe admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.284663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3527eb5-f09e-4794-a606-f3f3a5198ee3 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.324954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:06.339738 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06949e7-1925-4ded-b254-9a8aa25398fe admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.344662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d32e1-24e4-4440-b6ec-f808774b5c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.345439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d32e1-24e4-4440-b6ec-f808774b5c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.345716 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d32e1-24e4-4440-b6ec-f808774b5c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.346026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d32e1-24e4-4440-b6ec-f808774b5c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.346759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d32e1-24e4-4440-b6ec-f808774b5c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.347653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc2057-6a2b-4837-99c6-f6b87a2e747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.348316 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc2057-6a2b-4837-99c6-f6b87a2e747e admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.348429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d32e1-24e4-4440-b6ec-f808774b5c9a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.348600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc2057-6a2b-4837-99c6-f6b87a2e747e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.348877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc2057-6a2b-4837-99c6-f6b87a2e747e admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.349745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc2057-6a2b-4837-99c6-f6b87a2e747e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.351506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc2057-6a2b-4837-99c6-f6b87a2e747e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.360427 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1276/2524] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:06.364469 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d32e1-24e4-4440-b6ec-f808774b5c9a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.375875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d32e1-24e4-4440-b6ec-f808774b5c9a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.381271 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ff4ce-14af-4a33-94db-5f1ddfd5a8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.382426 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ff4ce-14af-4a33-94db-5f1ddfd5a8f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.382709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ff4ce-14af-4a33-94db-5f1ddfd5a8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.382982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ff4ce-14af-4a33-94db-5f1ddfd5a8f9 admin admin] PATH_INFO: `/v3/projects/6eddc743065649d19400019e9b59f743/users/f56db6ef86894b749bee8d2337991427/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.385070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ff4ce-14af-4a33-94db-5f1ddfd5a8f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eddc743065649d19400019e9b59f743, user_id=f56db6ef86894b749bee8d2337991427, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.385378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1249/2525] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:06 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:06.388577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ff4ce-14af-4a33-94db-5f1ddfd5a8f9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.390252 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc2057-6a2b-4837-99c6-f6b87a2e747e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.400777 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1277/2526] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:06 2022] PUT /identity/v3/projects/6eddc743065649d19400019e9b59f743/users/f56db6ef86894b749bee8d2337991427/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:51:06.415237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358ac50-b9be-495b-a184-c0757bfbdca6 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.415748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358ac50-b9be-495b-a184-c0757bfbdca6 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.416210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358ac50-b9be-495b-a184-c0757bfbdca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.451715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-5358ac50-b9be-495b-a184-c0757bfbdca6 None None] MFA Rules not processed for user `f56db6ef86894b749bee8d2337991427`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:06.452845 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5358ac50-b9be-495b-a184-c0757bfbdca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:06.463154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc2057-6a2b-4837-99c6-f6b87a2e747e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.480664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:06.526680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b120632-396e-4b7a-93dc-212530e29167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.527619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b120632-396e-4b7a-93dc-212530e29167 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.527744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1278/2527] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:06.527845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b120632-396e-4b7a-93dc-212530e29167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.528049 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b120632-396e-4b7a-93dc-212530e29167 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.528798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b120632-396e-4b7a-93dc-212530e29167 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.530778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b120632-396e-4b7a-93dc-212530e29167 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.539026 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358ac50-b9be-495b-a184-c0757bfbdca6 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.541161 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5358ac50-b9be-495b-a184-c0757bfbdca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:06.553800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1250/2528] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:06 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:06.558071 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b120632-396e-4b7a-93dc-212530e29167 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.567940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b120632-396e-4b7a-93dc-212530e29167 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.572864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a8075-4503-444c-ae62-5b819624a704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.573539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a8075-4503-444c-ae62-5b819624a704 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.573824 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a8075-4503-444c-ae62-5b819624a704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.574127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a8075-4503-444c-ae62-5b819624a704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.574852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a8075-4503-444c-ae62-5b819624a704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.576444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a8075-4503-444c-ae62-5b819624a704 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.588160 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1251/2529] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:51:06.592501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a8075-4503-444c-ae62-5b819624a704 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.603910 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358ac50-b9be-495b-a184-c0757bfbdca6 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.604480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a8075-4503-444c-ae62-5b819624a704 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.608203 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a16f83-7622-4d14-882e-02dd0e8d26d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.608861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a16f83-7622-4d14-882e-02dd0e8d26d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.609030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67556f02-b70b-4b75-8785-3a940d5eb755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.609168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a16f83-7622-4d14-882e-02dd0e8d26d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.609409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a16f83-7622-4d14-882e-02dd0e8d26d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.609693 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67556f02-b70b-4b75-8785-3a940d5eb755 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.610000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67556f02-b70b-4b75-8785-3a940d5eb755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.610274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67556f02-b70b-4b75-8785-3a940d5eb755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.610995 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67556f02-b70b-4b75-8785-3a940d5eb755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.612753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67556f02-b70b-4b75-8785-3a940d5eb755 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.619210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a16f83-7622-4d14-882e-02dd0e8d26d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.620677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a16f83-7622-4d14-882e-02dd0e8d26d9 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.624118 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1252/2530] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:51:06.628228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67556f02-b70b-4b75-8785-3a940d5eb755 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.631040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1279/2531] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:51:06.638665 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67556f02-b70b-4b75-8785-3a940d5eb755 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.643428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0eb51-44fe-43b3-94cb-db69272e6d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.644364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0eb51-44fe-43b3-94cb-db69272e6d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.644651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0eb51-44fe-43b3-94cb-db69272e6d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.644927 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0eb51-44fe-43b3-94cb-db69272e6d68 admin admin] PATH_INFO: `/v3/projects/64070e0911364917ab2419538b531c1c/users/12486f89f982414da66f5546eaa5d4e6/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.670485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0eb51-44fe-43b3-94cb-db69272e6d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64070e0911364917ab2419538b531c1c, user_id=12486f89f982414da66f5546eaa5d4e6, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.673135 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0eb51-44fe-43b3-94cb-db69272e6d68 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.686918 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1253/2532] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:06 2022] PUT /identity/v3/projects/64070e0911364917ab2419538b531c1c/users/12486f89f982414da66f5546eaa5d4e6/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:51:06.698099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7f270-1cd5-40f0-8e9c-e7c001356e80 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.698286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7f270-1cd5-40f0-8e9c-e7c001356e80 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.698567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7f270-1cd5-40f0-8e9c-e7c001356e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.714222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0eb51-44fe-43b3-94cb-db69272e6d68 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.733398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-fef7f270-1cd5-40f0-8e9c-e7c001356e80 None None] MFA Rules not processed for user `12486f89f982414da66f5546eaa5d4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:06.735450 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fef7f270-1cd5-40f0-8e9c-e7c001356e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:06.772720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0eb51-44fe-43b3-94cb-db69272e6d68 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.777142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863b144-1e5e-4bac-b421-bb192855a86a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.777803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863b144-1e5e-4bac-b421-bb192855a86a service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.778119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863b144-1e5e-4bac-b421-bb192855a86a service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.778402 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863b144-1e5e-4bac-b421-bb192855a86a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.780396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e863b144-1e5e-4bac-b421-bb192855a86a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:06.792082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:06.812813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1280/2533] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:06.817565 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7f270-1cd5-40f0-8e9c-e7c001356e80 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.854321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863b144-1e5e-4bac-b421-bb192855a86a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.855829 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863b144-1e5e-4bac-b421-bb192855a86a service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.866582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1254/2534] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:51:06.894752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7f270-1cd5-40f0-8e9c-e7c001356e80 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.902090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0213d6e-90ec-4671-9b32-6780689e55ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:06.902864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0213d6e-90ec-4671-9b32-6780689e55ec admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:06.903154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0213d6e-90ec-4671-9b32-6780689e55ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:06.903433 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0213d6e-90ec-4671-9b32-6780689e55ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:06.904167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0213d6e-90ec-4671-9b32-6780689e55ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:06.906236 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0213d6e-90ec-4671-9b32-6780689e55ec admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:06.939882 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1281/2535] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:06 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:06.944174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863b144-1e5e-4bac-b421-bb192855a86a service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:06.955783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863b144-1e5e-4bac-b421-bb192855a86a service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:06.970322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:07.031896 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fe272-6ad5-48d0-8726-64ecec28fbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:07.032835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fe272-6ad5-48d0-8726-64ecec28fbb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:07.033124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fe272-6ad5-48d0-8726-64ecec28fbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:07.033399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fe272-6ad5-48d0-8726-64ecec28fbb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:07.034125 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fe272-6ad5-48d0-8726-64ecec28fbb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:07.036226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fe272-6ad5-48d0-8726-64ecec28fbb7 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:07.058637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1255/2536] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:06 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:07.062969 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0213d6e-90ec-4671-9b32-6780689e55ec admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:07.074222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0213d6e-90ec-4671-9b32-6780689e55ec admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:07.078888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964b32f-1d7c-4ee1-8635-76633a2c5fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:07.079547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964b32f-1d7c-4ee1-8635-76633a2c5fce admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:07.079824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964b32f-1d7c-4ee1-8635-76633a2c5fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:07.080095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964b32f-1d7c-4ee1-8635-76633a2c5fce admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:07.080852 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964b32f-1d7c-4ee1-8635-76633a2c5fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:07.082634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964b32f-1d7c-4ee1-8635-76633a2c5fce admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:07.093885 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1282/2537] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:07.098229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fe272-6ad5-48d0-8726-64ecec28fbb7 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:07.109282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fe272-6ad5-48d0-8726-64ecec28fbb7 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:07.114023 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad9495-d3c6-4553-a179-5bc49fa34662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:07.114920 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad9495-d3c6-4553-a179-5bc49fa34662 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:07.115206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad9495-d3c6-4553-a179-5bc49fa34662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:07.115484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad9495-d3c6-4553-a179-5bc49fa34662 admin admin] PATH_INFO: `/v3/projects/39eae4b98b314c9a81ea99693bd56095/users/d8a62e82a73f4dd98cc66b385979d3fa/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:07.131753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad9495-d3c6-4553-a179-5bc49fa34662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eae4b98b314c9a81ea99693bd56095, user_id=d8a62e82a73f4dd98cc66b385979d3fa, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:07.134630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad9495-d3c6-4553-a179-5bc49fa34662 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:07.148713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1256/2538] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:07 2022] PUT /identity/v3/projects/39eae4b98b314c9a81ea99693bd56095/users/d8a62e82a73f4dd98cc66b385979d3fa/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:07.152968 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964b32f-1d7c-4ee1-8635-76633a2c5fce admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:07.163468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964b32f-1d7c-4ee1-8635-76633a2c5fce admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:07.239390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a20e4d-cb60-4ea2-9d7e-9eb0358e00a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:07.239971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a20e4d-cb60-4ea2-9d7e-9eb0358e00a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:07.240253 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a20e4d-cb60-4ea2-9d7e-9eb0358e00a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:07.240531 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a20e4d-cb60-4ea2-9d7e-9eb0358e00a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:07.241255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a20e4d-cb60-4ea2-9d7e-9eb0358e00a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:07.243540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a20e4d-cb60-4ea2-9d7e-9eb0358e00a3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:07.254076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1283/2539] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:07.258671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad9495-d3c6-4553-a179-5bc49fa34662 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:07.269671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad9495-d3c6-4553-a179-5bc49fa34662 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:07.274859 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd84297-6334-436b-be9f-8e85c30a7000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:07.275737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd84297-6334-436b-be9f-8e85c30a7000 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:07.276016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd84297-6334-436b-be9f-8e85c30a7000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:07.276289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd84297-6334-436b-be9f-8e85c30a7000 admin admin] PATH_INFO: `/v3/projects/39eae4b98b314c9a81ea99693bd56095/users/d8a62e82a73f4dd98cc66b385979d3fa/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:07.278300 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd84297-6334-436b-be9f-8e85c30a7000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eae4b98b314c9a81ea99693bd56095, user_id=d8a62e82a73f4dd98cc66b385979d3fa, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:07.280925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd84297-6334-436b-be9f-8e85c30a7000 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:07.292639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1257/2540] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:07 2022] PUT /identity/v3/projects/39eae4b98b314c9a81ea99693bd56095/users/d8a62e82a73f4dd98cc66b385979d3fa/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:07.304377 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac1000-3a72-4d1a-921f-a8a19c920958 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:07.304667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac1000-3a72-4d1a-921f-a8a19c920958 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:07.304954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac1000-3a72-4d1a-921f-a8a19c920958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:07.333528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-8bac1000-3a72-4d1a-921f-a8a19c920958 None None] MFA Rules not processed for user `d8a62e82a73f4dd98cc66b385979d3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:07.334788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8bac1000-3a72-4d1a-921f-a8a19c920958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:07.382762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:07.408055 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1284/2541] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:07.420053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd84297-6334-436b-be9f-8e85c30a7000 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:07.479166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd84297-6334-436b-be9f-8e85c30a7000 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:07.484515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7755f-fcd0-479c-8631-7d363bffec45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:07.485176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7755f-fcd0-479c-8631-7d363bffec45 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:07.485461 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7755f-fcd0-479c-8631-7d363bffec45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:07.485741 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7755f-fcd0-479c-8631-7d363bffec45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:07.495702 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7755f-fcd0-479c-8631-7d363bffec45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:07.497074 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7755f-fcd0-479c-8631-7d363bffec45 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:07.507414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1258/2542] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:07.588641 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac1000-3a72-4d1a-921f-a8a19c920958 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:07.603427 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac1000-3a72-4d1a-921f-a8a19c920958 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:07.607126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d448a6d-139f-40dd-94ac-e7a4ce200784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:07.608036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d448a6d-139f-40dd-94ac-e7a4ce200784 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:07.608318 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d448a6d-139f-40dd-94ac-e7a4ce200784 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:07.608595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d448a6d-139f-40dd-94ac-e7a4ce200784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:07.665987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d448a6d-139f-40dd-94ac-e7a4ce200784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:07.667446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d448a6d-139f-40dd-94ac-e7a4ce200784 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:07.678356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1285/2543] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:09.653318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7755f-fcd0-479c-8631-7d363bffec45 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:09.719633 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7755f-fcd0-479c-8631-7d363bffec45 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:09.724621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:09.725494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:09.725780 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:09.726091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] PATH_INFO: `/v3/users/cae13ad7a42644c4be9b792e51b12e09` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:09.727878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae13ad7a42644c4be9b792e51b12e09)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:09.729837 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:09.744414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:09.778956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae13ad7a42644c4be9b792e51b12e09'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:09.779619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae13ad7a42644c4be9b792e51b12e09'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:09.780231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae13ad7a42644c4be9b792e51b12e09'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:09.787243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae13ad7a42644c4be9b792e51b12e09'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:09.796010 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae13ad7a42644c4be9b792e51b12e09'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:09.814147 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1259/2544] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:09 2022] DELETE /identity/v3/users/cae13ad7a42644c4be9b792e51b12e09 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:09.818826 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230564c-51fa-43ec-ac9c-c523efa41335 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:09.819347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230564c-51fa-43ec-ac9c-c523efa41335 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:09.819445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230564c-51fa-43ec-ac9c-c523efa41335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:09.839224 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0230564c-51fa-43ec-ac9c-c523efa41335 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:09.840432 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0230564c-51fa-43ec-ac9c-c523efa41335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:09.900892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1286/2545] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:09.910902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:09.912436 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:09.970563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8523c-549d-4971-acdb-5eb699741f00 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:09.974299 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72908203-9f55-4db0-be39-6dd439a5c43a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:09.974959 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72908203-9f55-4db0-be39-6dd439a5c43a service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:09.975246 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72908203-9f55-4db0-be39-6dd439a5c43a service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:09.975523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72908203-9f55-4db0-be39-6dd439a5c43a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:09.985175 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72908203-9f55-4db0-be39-6dd439a5c43a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:09.986645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72908203-9f55-4db0-be39-6dd439a5c43a service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:09.996722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1260/2546] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:10.292867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230564c-51fa-43ec-ac9c-c523efa41335 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:10.298137 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0230564c-51fa-43ec-ac9c-c523efa41335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:10.315914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230564c-51fa-43ec-ac9c-c523efa41335 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:10.329191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:10.329191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:10.329191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:10.329191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] PATH_INFO: `/v3/projects/2e17abf1718a4b5eaa8c31d6fd0c8af8` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:10.329962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e17abf1718a4b5eaa8c31d6fd0c8af8)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:10.330652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:10.382082 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e17abf1718a4b5eaa8c31d6fd0c8af8', 'request_id': 'req-fd38531c-c55a-45c7-b850-22d0e75589a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:10.390959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e17abf1718a4b5eaa8c31d6fd0c8af8', 'request_id': 'req-fd38531c-c55a-45c7-b850-22d0e75589a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:10.398977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e17abf1718a4b5eaa8c31d6fd0c8af8', 'request_id': 'req-fd38531c-c55a-45c7-b850-22d0e75589a3'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:10.409898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] The token cache is being invalidate because project 2e17abf1718a4b5eaa8c31d6fd0c8af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:10.410128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:10.410803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:10.412374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1287/2547] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:10 2022] DELETE /identity/v3/projects/2e17abf1718a4b5eaa8c31d6fd0c8af8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:10.416506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72908203-9f55-4db0-be39-6dd439a5c43a service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:10.428045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72908203-9f55-4db0-be39-6dd439a5c43a service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:10.503412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d96ce-35a8-4c7b-941d-5e53dbe837f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:10.504527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d96ce-35a8-4c7b-941d-5e53dbe837f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:10.505026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d96ce-35a8-4c7b-941d-5e53dbe837f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:10.505390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d96ce-35a8-4c7b-941d-5e53dbe837f8 admin admin] PATH_INFO: `/v3/users/f56db6ef86894b749bee8d2337991427` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:10.507013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d96ce-35a8-4c7b-941d-5e53dbe837f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56db6ef86894b749bee8d2337991427)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:10.510163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d96ce-35a8-4c7b-941d-5e53dbe837f8 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:10.557165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:10.569712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-108d96ce-35a8-4c7b-941d-5e53dbe837f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56db6ef86894b749bee8d2337991427'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:10.570634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-108d96ce-35a8-4c7b-941d-5e53dbe837f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56db6ef86894b749bee8d2337991427'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:10.572563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-108d96ce-35a8-4c7b-941d-5e53dbe837f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56db6ef86894b749bee8d2337991427'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:10.593443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-108d96ce-35a8-4c7b-941d-5e53dbe837f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56db6ef86894b749bee8d2337991427'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:10.603936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-108d96ce-35a8-4c7b-941d-5e53dbe837f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56db6ef86894b749bee8d2337991427'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:10.624466 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1261/2548] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:10 2022] DELETE /identity/v3/users/f56db6ef86894b749bee8d2337991427 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:10.907930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:10.909453 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:10.925359 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38531c-c55a-45c7-b850-22d0e75589a3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:10.929531 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:10.986805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:10.987677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:10.987972 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:10.989751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae admin admin] PATH_INFO: `/v3/projects/6eddc743065649d19400019e9b59f743` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:10.989751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eddc743065649d19400019e9b59f743)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:10.991546 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:11.041112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eddc743065649d19400019e9b59f743', 'request_id': 'req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:11.049537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eddc743065649d19400019e9b59f743', 'request_id': 'req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:11.056322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eddc743065649d19400019e9b59f743', 'request_id': 'req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:11.067123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae admin admin] The token cache is being invalidate because project 6eddc743065649d19400019e9b59f743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:11.067407 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:11.068060 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-255b2bf3-e20e-4d2c-8475-53faf0ebd3ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:11.069608 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1288/2549] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:10 2022] DELETE /identity/v3/projects/6eddc743065649d19400019e9b59f743 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:11.076381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5ab48-a84f-4ea0-a190-5df81d751e97 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.076863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5ab48-a84f-4ea0-a190-5df81d751e97 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.076982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5ab48-a84f-4ea0-a190-5df81d751e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.099190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-8ff5ab48-a84f-4ea0-a190-5df81d751e97 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:11.100427 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8ff5ab48-a84f-4ea0-a190-5df81d751e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:11.163753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1262/2550] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:51:11.172589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be532a7-fbf7-4964-8290-d5bc0a3af4b9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.172974 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be532a7-fbf7-4964-8290-d5bc0a3af4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.173261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be532a7-fbf7-4964-8290-d5bc0a3af4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.195311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9be532a7-fbf7-4964-8290-d5bc0a3af4b9 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:11.196537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9be532a7-fbf7-4964-8290-d5bc0a3af4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:11.204567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1289/2551] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:11.209078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5ab48-a84f-4ea0-a190-5df81d751e97 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:11.219550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5ab48-a84f-4ea0-a190-5df81d751e97 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:11.224211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce241e-fdc8-4bbc-a281-6bd4e62b69ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:11.224879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce241e-fdc8-4bbc-a281-6bd4e62b69ad admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.225165 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce241e-fdc8-4bbc-a281-6bd4e62b69ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.225441 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce241e-fdc8-4bbc-a281-6bd4e62b69ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.226223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce241e-fdc8-4bbc-a281-6bd4e62b69ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:51:11.226534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce241e-fdc8-4bbc-a281-6bd4e62b69ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:11.228775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce241e-fdc8-4bbc-a281-6bd4e62b69ad admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:11.242211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1263/2552] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:51:11.245732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be532a7-fbf7-4964-8290-d5bc0a3af4b9 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:11.256399 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be532a7-fbf7-4964-8290-d5bc0a3af4b9 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:11.261970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e06b3-85b6-46ca-9c2d-cdb5c8709754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:11.262892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e06b3-85b6-46ca-9c2d-cdb5c8709754 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.263221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e06b3-85b6-46ca-9c2d-cdb5c8709754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.263494 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e06b3-85b6-46ca-9c2d-cdb5c8709754 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.264221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e06b3-85b6-46ca-9c2d-cdb5c8709754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:11.266270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e06b3-85b6-46ca-9c2d-cdb5c8709754 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:11.297663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1290/2553] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:11 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:11.298829 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce241e-fdc8-4bbc-a281-6bd4e62b69ad admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:11.308953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce241e-fdc8-4bbc-a281-6bd4e62b69ad admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:11.366558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82578684-6858-45c7-95f4-688ab97ff6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:11.367226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82578684-6858-45c7-95f4-688ab97ff6ea admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.367507 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82578684-6858-45c7-95f4-688ab97ff6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.367782 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82578684-6858-45c7-95f4-688ab97ff6ea admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.368503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82578684-6858-45c7-95f4-688ab97ff6ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:11.370466 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82578684-6858-45c7-95f4-688ab97ff6ea admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:11.392630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1264/2554] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:11 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:11.396974 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e06b3-85b6-46ca-9c2d-cdb5c8709754 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:11.408128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e06b3-85b6-46ca-9c2d-cdb5c8709754 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:11.412789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52128d-d6f3-4288-bc09-590686730f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:11.413449 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52128d-d6f3-4288-bc09-590686730f68 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.413727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52128d-d6f3-4288-bc09-590686730f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.414032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52128d-d6f3-4288-bc09-590686730f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.414794 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52128d-d6f3-4288-bc09-590686730f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:11.416404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52128d-d6f3-4288-bc09-590686730f68 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:11.427339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1291/2555] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:11.431629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82578684-6858-45c7-95f4-688ab97ff6ea admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:11.442096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82578684-6858-45c7-95f4-688ab97ff6ea admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:11.447296 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a7bf2-ca8a-4eff-ab3f-08569aca35f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:11.447989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a7bf2-ca8a-4eff-ab3f-08569aca35f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.448275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a7bf2-ca8a-4eff-ab3f-08569aca35f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.448552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a7bf2-ca8a-4eff-ab3f-08569aca35f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.449292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a7bf2-ca8a-4eff-ab3f-08569aca35f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:11.450951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a7bf2-ca8a-4eff-ab3f-08569aca35f7 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:11.461492 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1265/2556] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:11.466404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52128d-d6f3-4288-bc09-590686730f68 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:11.476651 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52128d-d6f3-4288-bc09-590686730f68 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:11.481280 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c7c9c-6fb8-4012-8b53-84afdb47ec14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:11.482268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c7c9c-6fb8-4012-8b53-84afdb47ec14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.482810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c7c9c-6fb8-4012-8b53-84afdb47ec14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.483136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c7c9c-6fb8-4012-8b53-84afdb47ec14 admin admin] PATH_INFO: `/v3/projects/186a5a4594e843bc839f41bb8d718035/users/abf139353d7a45b59c1685f93cc434e6/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.499712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c7c9c-6fb8-4012-8b53-84afdb47ec14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=186a5a4594e843bc839f41bb8d718035, user_id=abf139353d7a45b59c1685f93cc434e6, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:11.502410 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c7c9c-6fb8-4012-8b53-84afdb47ec14 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:11.516275 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1292/2557] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:11 2022] PUT /identity/v3/projects/186a5a4594e843bc839f41bb8d718035/users/abf139353d7a45b59c1685f93cc434e6/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:11.529013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10566cde-155b-423e-bdcf-11351d961fb4 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.529305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10566cde-155b-423e-bdcf-11351d961fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.529605 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10566cde-155b-423e-bdcf-11351d961fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.558648 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-10566cde-155b-423e-bdcf-11351d961fb4 None None] MFA Rules not processed for user `abf139353d7a45b59c1685f93cc434e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:11.559886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-10566cde-155b-423e-bdcf-11351d961fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:11.568582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:11.632655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1266/2558] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:11.637830 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c7c9c-6fb8-4012-8b53-84afdb47ec14 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:11.674193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:11.708242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c7c9c-6fb8-4012-8b53-84afdb47ec14 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:11.713547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c80028-017d-42c8-95b2-4c7ccf482662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:11.714261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c80028-017d-42c8-95b2-4c7ccf482662 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.714548 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c80028-017d-42c8-95b2-4c7ccf482662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.714967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c80028-017d-42c8-95b2-4c7ccf482662 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.715677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c80028-017d-42c8-95b2-4c7ccf482662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:11.717601 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c80028-017d-42c8-95b2-4c7ccf482662 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:11.750551 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1293/2559] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:11 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:11.756244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10566cde-155b-423e-bdcf-11351d961fb4 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:11.771035 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10566cde-155b-423e-bdcf-11351d961fb4 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:11.834327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4e0d6-7f56-4a8d-b9e4-c7a777260b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:11.835001 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4e0d6-7f56-4a8d-b9e4-c7a777260b02 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.835289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4e0d6-7f56-4a8d-b9e4-c7a777260b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.835564 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4e0d6-7f56-4a8d-b9e4-c7a777260b02 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.836279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4e0d6-7f56-4a8d-b9e4-c7a777260b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:11.838275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4e0d6-7f56-4a8d-b9e4-c7a777260b02 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:11.864245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1267/2560] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:11 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:11.866513 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c80028-017d-42c8-95b2-4c7ccf482662 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:11.877614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c80028-017d-42c8-95b2-4c7ccf482662 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:11.882324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab514c1-8e15-4328-8a81-8bc440778a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:11.882987 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab514c1-8e15-4328-8a81-8bc440778a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.883266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab514c1-8e15-4328-8a81-8bc440778a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.884085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab514c1-8e15-4328-8a81-8bc440778a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.884814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab514c1-8e15-4328-8a81-8bc440778a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:11.887681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab514c1-8e15-4328-8a81-8bc440778a9e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:11.897390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1294/2561] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:11.902061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4e0d6-7f56-4a8d-b9e4-c7a777260b02 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:11.912331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4e0d6-7f56-4a8d-b9e4-c7a777260b02 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:11.917031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c29a33-d2d9-4fda-a38e-b09d8742b575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:11.917940 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c29a33-d2d9-4fda-a38e-b09d8742b575 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:11.918231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c29a33-d2d9-4fda-a38e-b09d8742b575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:11.918507 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c29a33-d2d9-4fda-a38e-b09d8742b575 admin admin] PATH_INFO: `/v3/projects/79733868ac424f51bf436a09de5c1513/users/bcb1e11236bd4b248adbdebe3175995a/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:11.934381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c29a33-d2d9-4fda-a38e-b09d8742b575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79733868ac424f51bf436a09de5c1513, user_id=bcb1e11236bd4b248adbdebe3175995a, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:11.937058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c29a33-d2d9-4fda-a38e-b09d8742b575 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:11.953084 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1268/2562] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:11 2022] PUT /identity/v3/projects/79733868ac424f51bf436a09de5c1513/users/bcb1e11236bd4b248adbdebe3175995a/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:11.963696 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab514c1-8e15-4328-8a81-8bc440778a9e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:11.973956 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab514c1-8e15-4328-8a81-8bc440778a9e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:12.031176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f6ee1-c3c8-4e3f-a184-d0c30f7ca625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:12.031850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6ee1-c3c8-4e3f-a184-d0c30f7ca625 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:12.032134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6ee1-c3c8-4e3f-a184-d0c30f7ca625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:12.032413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6ee1-c3c8-4e3f-a184-d0c30f7ca625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:12.033148 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f6ee1-c3c8-4e3f-a184-d0c30f7ca625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:12.034788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f6ee1-c3c8-4e3f-a184-d0c30f7ca625 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:12.044997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1295/2563] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:12.049331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c29a33-d2d9-4fda-a38e-b09d8742b575 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:12.060351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c29a33-d2d9-4fda-a38e-b09d8742b575 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:12.064913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72095532-5443-4e9e-a204-033c4030543d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:12.065793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72095532-5443-4e9e-a204-033c4030543d admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:12.066106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72095532-5443-4e9e-a204-033c4030543d admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:12.066377 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72095532-5443-4e9e-a204-033c4030543d admin admin] PATH_INFO: `/v3/projects/79733868ac424f51bf436a09de5c1513/users/bcb1e11236bd4b248adbdebe3175995a/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:12.068349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72095532-5443-4e9e-a204-033c4030543d admin admin] RBAC: Authorizing `identity:create_grant(project_id=79733868ac424f51bf436a09de5c1513, user_id=bcb1e11236bd4b248adbdebe3175995a, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:12.071616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72095532-5443-4e9e-a204-033c4030543d admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:12.082236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1269/2564] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:12 2022] PUT /identity/v3/projects/79733868ac424f51bf436a09de5c1513/users/bcb1e11236bd4b248adbdebe3175995a/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:12.093347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989a77-ac3c-454b-a224-5fbdff944892 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:12.093683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989a77-ac3c-454b-a224-5fbdff944892 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:12.093994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989a77-ac3c-454b-a224-5fbdff944892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:12.124255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-28989a77-ac3c-454b-a224-5fbdff944892 None None] MFA Rules not processed for user `bcb1e11236bd4b248adbdebe3175995a`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:12.125544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-28989a77-ac3c-454b-a224-5fbdff944892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:12.196529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1296/2565] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:12.207963 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72095532-5443-4e9e-a204-033c4030543d admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:12.209731 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-72095532-5443-4e9e-a204-033c4030543d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:12.273228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72095532-5443-4e9e-a204-033c4030543d admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:12.276964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ae7cc-4e38-44c2-a113-22ce2dd88fdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:12.277622 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ae7cc-4e38-44c2-a113-22ce2dd88fdd service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:12.277935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ae7cc-4e38-44c2-a113-22ce2dd88fdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:12.278216 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ae7cc-4e38-44c2-a113-22ce2dd88fdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:12.287953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ae7cc-4e38-44c2-a113-22ce2dd88fdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:12.289412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ae7cc-4e38-44c2-a113-22ce2dd88fdd service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:12.300116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1270/2566] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:51:12.389074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28989a77-ac3c-454b-a224-5fbdff944892 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:12.399497 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28989a77-ac3c-454b-a224-5fbdff944892 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:12.403967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d33ee-953d-48bf-a64c-d5219df3fdef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:12.404626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d33ee-953d-48bf-a64c-d5219df3fdef service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:12.404910 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d33ee-953d-48bf-a64c-d5219df3fdef service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:12.405188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d33ee-953d-48bf-a64c-d5219df3fdef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:12.433995 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:12.466228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d33ee-953d-48bf-a64c-d5219df3fdef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:12.467681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d33ee-953d-48bf-a64c-d5219df3fdef service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:12.480122 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1297/2567] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:19.418784 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ae7cc-4e38-44c2-a113-22ce2dd88fdd service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:19.424159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:19.487447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ae7cc-4e38-44c2-a113-22ce2dd88fdd service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:19.492406 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:19.493391 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:19.493810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:19.494181 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] PATH_INFO: `/v3/users/abf139353d7a45b59c1685f93cc434e6` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:19.495552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf139353d7a45b59c1685f93cc434e6)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:19.497812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:19.545653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf139353d7a45b59c1685f93cc434e6'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:19.546379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf139353d7a45b59c1685f93cc434e6'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:19.547053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf139353d7a45b59c1685f93cc434e6'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:19.554411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf139353d7a45b59c1685f93cc434e6'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:19.562129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf139353d7a45b59c1685f93cc434e6'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:19.582040 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1271/2568] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:19 2022] DELETE /identity/v3/users/abf139353d7a45b59c1685f93cc434e6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:19.587505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f03a49-5c99-4227-af2a-48665be173a2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:19.588015 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f03a49-5c99-4227-af2a-48665be173a2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:19.588113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f03a49-5c99-4227-af2a-48665be173a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:19.608559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-42f03a49-5c99-4227-af2a-48665be173a2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:19.609790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-42f03a49-5c99-4227-af2a-48665be173a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:19.674944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1298/2569] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:19.684820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:19.686286 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:19.744509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42838de7-96eb-47b3-ba6f-cd51637ab303 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:19.748206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41558300-4a8d-47af-b52c-419224d7618b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:19.748860 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41558300-4a8d-47af-b52c-419224d7618b service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:19.749143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41558300-4a8d-47af-b52c-419224d7618b service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:19.749421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41558300-4a8d-47af-b52c-419224d7618b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:19.765179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41558300-4a8d-47af-b52c-419224d7618b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:19.766655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41558300-4a8d-47af-b52c-419224d7618b service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:19.777567 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1272/2570] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:20.140858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f03a49-5c99-4227-af2a-48665be173a2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:20.142472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-42f03a49-5c99-4227-af2a-48665be173a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:20.155734 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f03a49-5c99-4227-af2a-48665be173a2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:20.163799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:20.164818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:20.165103 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:20.165471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] PATH_INFO: `/v3/projects/186a5a4594e843bc839f41bb8d718035` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:20.166888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] RBAC: Authorizing `identity:delete_project(project_id=186a5a4594e843bc839f41bb8d718035)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:20.168874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:20.242004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '186a5a4594e843bc839f41bb8d718035', 'request_id': 'req-68065817-6129-4966-ad44-c7159c90c92a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:20.251486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '186a5a4594e843bc839f41bb8d718035', 'request_id': 'req-68065817-6129-4966-ad44-c7159c90c92a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:20.260393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '186a5a4594e843bc839f41bb8d718035', 'request_id': 'req-68065817-6129-4966-ad44-c7159c90c92a'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:20.272867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] The token cache is being invalidate because project 186a5a4594e843bc839f41bb8d718035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:20.273160 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:20.273827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:20.276294 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1299/2571] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:20 2022] DELETE /identity/v3/projects/186a5a4594e843bc839f41bb8d718035 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:20.280192 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41558300-4a8d-47af-b52c-419224d7618b service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:20.293847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41558300-4a8d-47af-b52c-419224d7618b service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:20.325247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:20.371607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0b9cb-9b54-405e-a916-39fd185960f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:20.372476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0b9cb-9b54-405e-a916-39fd185960f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:20.372911 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0b9cb-9b54-405e-a916-39fd185960f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:20.373253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0b9cb-9b54-405e-a916-39fd185960f0 admin admin] PATH_INFO: `/v3/users/bcb1e11236bd4b248adbdebe3175995a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:20.374729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0b9cb-9b54-405e-a916-39fd185960f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb1e11236bd4b248adbdebe3175995a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:20.377085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0b9cb-9b54-405e-a916-39fd185960f0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:20.438992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1bc0b9cb-9b54-405e-a916-39fd185960f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb1e11236bd4b248adbdebe3175995a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:20.439809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1bc0b9cb-9b54-405e-a916-39fd185960f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb1e11236bd4b248adbdebe3175995a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:20.440676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1bc0b9cb-9b54-405e-a916-39fd185960f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb1e11236bd4b248adbdebe3175995a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:20.449030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1bc0b9cb-9b54-405e-a916-39fd185960f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb1e11236bd4b248adbdebe3175995a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:20.460445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-1bc0b9cb-9b54-405e-a916-39fd185960f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb1e11236bd4b248adbdebe3175995a'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:20.487291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1273/2572] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:20 2022] DELETE /identity/v3/users/bcb1e11236bd4b248adbdebe3175995a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:21.044501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:21.045997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:21.060977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68065817-6129-4966-ad44-c7159c90c92a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:21.110157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:21.121787 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:21.122732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:21.123240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:21.123521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8 admin admin] PATH_INFO: `/v3/projects/79733868ac424f51bf436a09de5c1513` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:21.124748 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79733868ac424f51bf436a09de5c1513)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:21.126745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:21.173045 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79733868ac424f51bf436a09de5c1513', 'request_id': 'req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:21.181878 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79733868ac424f51bf436a09de5c1513', 'request_id': 'req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:21.188525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79733868ac424f51bf436a09de5c1513', 'request_id': 'req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:21.198932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8 admin admin] The token cache is being invalidate because project 79733868ac424f51bf436a09de5c1513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:21.199214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:21.199854 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-b1b33d8c-d63f-49a2-9ee7-0e7116226cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:21.201385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1300/2573] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:21 2022] DELETE /identity/v3/projects/79733868ac424f51bf436a09de5c1513 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:51:21.206493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab7f9-d11e-4d02-8dd7-04512b598547 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:21.206787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab7f9-d11e-4d02-8dd7-04512b598547 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:21.207091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab7f9-d11e-4d02-8dd7-04512b598547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:21.228428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-76cab7f9-d11e-4d02-8dd7-04512b598547 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:21.229637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-76cab7f9-d11e-4d02-8dd7-04512b598547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:21.292322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1274/2574] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:21.300672 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366639e-f686-471f-bb6a-94540f6edd16 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:21.301003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366639e-f686-471f-bb6a-94540f6edd16 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:21.301349 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366639e-f686-471f-bb6a-94540f6edd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:21.328288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a366639e-f686-471f-bb6a-94540f6edd16 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:21.329574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a366639e-f686-471f-bb6a-94540f6edd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:21.338963 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1301/2575] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:21.343606 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab7f9-d11e-4d02-8dd7-04512b598547 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:21.354389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab7f9-d11e-4d02-8dd7-04512b598547 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:21.360440 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeedb64-78d4-47fa-bb73-886c99d60c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:21.361166 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeedb64-78d4-47fa-bb73-886c99d60c6b admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:21.361494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeedb64-78d4-47fa-bb73-886c99d60c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:21.362239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeedb64-78d4-47fa-bb73-886c99d60c6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:21.363206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeedb64-78d4-47fa-bb73-886c99d60c6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:51:21.363522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeedb64-78d4-47fa-bb73-886c99d60c6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:21.365325 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeedb64-78d4-47fa-bb73-886c99d60c6b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:21.379146 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1275/2576] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:51:21.384715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366639e-f686-471f-bb6a-94540f6edd16 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:21.395903 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366639e-f686-471f-bb6a-94540f6edd16 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:21.400592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57cc33-04fc-46c1-8ada-b5eda97943e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:21.401254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57cc33-04fc-46c1-8ada-b5eda97943e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:21.401538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57cc33-04fc-46c1-8ada-b5eda97943e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:21.401817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57cc33-04fc-46c1-8ada-b5eda97943e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:21.402546 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57cc33-04fc-46c1-8ada-b5eda97943e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:21.404468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57cc33-04fc-46c1-8ada-b5eda97943e2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:21.452665 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1302/2577] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:21 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:21.454188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeedb64-78d4-47fa-bb73-886c99d60c6b admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:21.465176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeedb64-78d4-47fa-bb73-886c99d60c6b admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:21.501381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:21.534603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d44dc-33ea-40bb-8880-509b2f0041d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:21.535512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d44dc-33ea-40bb-8880-509b2f0041d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:21.535814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d44dc-33ea-40bb-8880-509b2f0041d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:21.536072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d44dc-33ea-40bb-8880-509b2f0041d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:21.536787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d44dc-33ea-40bb-8880-509b2f0041d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:21.538934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d44dc-33ea-40bb-8880-509b2f0041d7 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:21.561608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1276/2578] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:21 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:21.565898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57cc33-04fc-46c1-8ada-b5eda97943e2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:21.577127 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57cc33-04fc-46c1-8ada-b5eda97943e2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:21.582221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cef8d4-57ba-4b7d-916f-0c83f32e7530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:21.582908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cef8d4-57ba-4b7d-916f-0c83f32e7530 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:21.583185 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cef8d4-57ba-4b7d-916f-0c83f32e7530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:21.583477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cef8d4-57ba-4b7d-916f-0c83f32e7530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:21.584197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cef8d4-57ba-4b7d-916f-0c83f32e7530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:21.585835 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cef8d4-57ba-4b7d-916f-0c83f32e7530 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:21.598002 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1303/2579] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:21.602227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d44dc-33ea-40bb-8880-509b2f0041d7 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:21.613803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d44dc-33ea-40bb-8880-509b2f0041d7 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:21.619046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f2bb8-9c76-4283-99b6-ea66a6c110a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:21.619725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f2bb8-9c76-4283-99b6-ea66a6c110a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:21.620004 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f2bb8-9c76-4283-99b6-ea66a6c110a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:21.620311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f2bb8-9c76-4283-99b6-ea66a6c110a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:21.621042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f2bb8-9c76-4283-99b6-ea66a6c110a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:21.622688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f2bb8-9c76-4283-99b6-ea66a6c110a2 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:21.635241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1277/2580] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:21.640220 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cef8d4-57ba-4b7d-916f-0c83f32e7530 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:21.652454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cef8d4-57ba-4b7d-916f-0c83f32e7530 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:21.657209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776556c-b2a3-433b-a004-112b73a923ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:21.658168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776556c-b2a3-433b-a004-112b73a923ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:21.658452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776556c-b2a3-433b-a004-112b73a923ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:21.658728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776556c-b2a3-433b-a004-112b73a923ce admin admin] PATH_INFO: `/v3/projects/0de3d2e81d0a438bb2257d3f430e50d9/users/5b916df4be3d4ca1bb1f5166f7271824/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:21.674666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776556c-b2a3-433b-a004-112b73a923ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de3d2e81d0a438bb2257d3f430e50d9, user_id=5b916df4be3d4ca1bb1f5166f7271824, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:21.677343 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776556c-b2a3-433b-a004-112b73a923ce admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:21.691624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1304/2581] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:21 2022] PUT /identity/v3/projects/0de3d2e81d0a438bb2257d3f430e50d9/users/5b916df4be3d4ca1bb1f5166f7271824/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:21.761937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec90da-7a4e-4214-98a0-0219408c5216 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:21.762319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec90da-7a4e-4214-98a0-0219408c5216 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:21.762498 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec90da-7a4e-4214-98a0-0219408c5216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:21.802410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-44ec90da-7a4e-4214-98a0-0219408c5216 None None] MFA Rules not processed for user `5b916df4be3d4ca1bb1f5166f7271824`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:21.803706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-44ec90da-7a4e-4214-98a0-0219408c5216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:21.894075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1278/2582] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:51:21.896215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776556c-b2a3-433b-a004-112b73a923ce admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:21.968487 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776556c-b2a3-433b-a004-112b73a923ce admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:21.973261 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f603704-0e01-4526-b63b-2acc89aeb195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:21.974074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f603704-0e01-4526-b63b-2acc89aeb195 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:21.974355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f603704-0e01-4526-b63b-2acc89aeb195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:21.974674 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f603704-0e01-4526-b63b-2acc89aeb195 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:21.975375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f603704-0e01-4526-b63b-2acc89aeb195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:21.977284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f603704-0e01-4526-b63b-2acc89aeb195 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:22.015418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1305/2583] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:21 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:22.016245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec90da-7a4e-4214-98a0-0219408c5216 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:22.029242 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec90da-7a4e-4214-98a0-0219408c5216 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:22.104859 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b5086-e94e-4ce7-96ae-f4df2b43c46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:22.105663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b5086-e94e-4ce7-96ae-f4df2b43c46c admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:22.105977 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b5086-e94e-4ce7-96ae-f4df2b43c46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:22.106364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b5086-e94e-4ce7-96ae-f4df2b43c46c admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:22.107437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b5086-e94e-4ce7-96ae-f4df2b43c46c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:22.111720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b5086-e94e-4ce7-96ae-f4df2b43c46c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:22.140420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1279/2584] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:22 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:22.144075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f603704-0e01-4526-b63b-2acc89aeb195 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:22.152398 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:22.160788 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f603704-0e01-4526-b63b-2acc89aeb195 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:22.178493 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4d1fa-d532-4cf3-84ac-55c3f7d2c77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:22.179222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4d1fa-d532-4cf3-84ac-55c3f7d2c77d admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:22.179511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4d1fa-d532-4cf3-84ac-55c3f7d2c77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:22.179897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4d1fa-d532-4cf3-84ac-55c3f7d2c77d admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:22.180771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4d1fa-d532-4cf3-84ac-55c3f7d2c77d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:22.182452 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4d1fa-d532-4cf3-84ac-55c3f7d2c77d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:22.209345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1306/2585] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:22.210248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b5086-e94e-4ce7-96ae-f4df2b43c46c admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:22.221745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b5086-e94e-4ce7-96ae-f4df2b43c46c admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:22.226549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72313a9-ceac-41de-b623-3adc04483077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:22.227236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72313a9-ceac-41de-b623-3adc04483077 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:22.227522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72313a9-ceac-41de-b623-3adc04483077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:22.227852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72313a9-ceac-41de-b623-3adc04483077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:22.228588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72313a9-ceac-41de-b623-3adc04483077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:22.230298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72313a9-ceac-41de-b623-3adc04483077 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:22.241532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1280/2586] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:51:22.246526 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4d1fa-d532-4cf3-84ac-55c3f7d2c77d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:22.257300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4d1fa-d532-4cf3-84ac-55c3f7d2c77d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:22.263895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c97c36-52fd-421e-acda-734b5c8a1a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:22.264804 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c97c36-52fd-421e-acda-734b5c8a1a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:22.265945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c97c36-52fd-421e-acda-734b5c8a1a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:22.265945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c97c36-52fd-421e-acda-734b5c8a1a34 admin admin] PATH_INFO: `/v3/projects/f7a3e29c257447f6b83c389643a0ed70/users/85320c1ed47545e0873f46a1cdc06d68/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:22.283302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c97c36-52fd-421e-acda-734b5c8a1a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a3e29c257447f6b83c389643a0ed70, user_id=85320c1ed47545e0873f46a1cdc06d68, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:22.286184 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c97c36-52fd-421e-acda-734b5c8a1a34 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:22.301081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1307/2587] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:22 2022] PUT /identity/v3/projects/f7a3e29c257447f6b83c389643a0ed70/users/85320c1ed47545e0873f46a1cdc06d68/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:22.312095 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d309a-944c-4f3f-a06a-8289224e7622 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:22.312385 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d309a-944c-4f3f-a06a-8289224e7622 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:22.312658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d309a-944c-4f3f-a06a-8289224e7622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:22.343034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-8e8d309a-944c-4f3f-a06a-8289224e7622 None None] MFA Rules not processed for user `85320c1ed47545e0873f46a1cdc06d68`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:22.344255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8e8d309a-944c-4f3f-a06a-8289224e7622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:22.352475 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:22.410848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1281/2588] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:51:22.415186 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c97c36-52fd-421e-acda-734b5c8a1a34 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:22.476269 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c97c36-52fd-421e-acda-734b5c8a1a34 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:22.480818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4070aa-9938-4642-be04-babf88e33ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:22.481475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4070aa-9938-4642-be04-babf88e33ecd admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:22.481770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4070aa-9938-4642-be04-babf88e33ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:22.482068 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4070aa-9938-4642-be04-babf88e33ecd admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:22.483037 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4070aa-9938-4642-be04-babf88e33ecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:22.484944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4070aa-9938-4642-be04-babf88e33ecd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:22.512628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1308/2589] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:22 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:22.517087 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d309a-944c-4f3f-a06a-8289224e7622 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:22.528483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d309a-944c-4f3f-a06a-8289224e7622 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:22.587141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b84e5-66b0-49c8-8a2e-727c9180d9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:22.587828 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b84e5-66b0-49c8-8a2e-727c9180d9da admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:22.588110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b84e5-66b0-49c8-8a2e-727c9180d9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:22.588392 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b84e5-66b0-49c8-8a2e-727c9180d9da admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:22.589111 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b84e5-66b0-49c8-8a2e-727c9180d9da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:22.591088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b84e5-66b0-49c8-8a2e-727c9180d9da admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:22.614616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1282/2590] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:22 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:22.619447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4070aa-9938-4642-be04-babf88e33ecd admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:22.633090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4070aa-9938-4642-be04-babf88e33ecd admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:22.644035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83614cf2-e6ef-4a22-b42a-17d69113e8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:22.644035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83614cf2-e6ef-4a22-b42a-17d69113e8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:22.644035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83614cf2-e6ef-4a22-b42a-17d69113e8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:22.644035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83614cf2-e6ef-4a22-b42a-17d69113e8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:22.644496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83614cf2-e6ef-4a22-b42a-17d69113e8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:22.646352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83614cf2-e6ef-4a22-b42a-17d69113e8f8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:22.667618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1309/2591] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:22.671844 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b84e5-66b0-49c8-8a2e-727c9180d9da admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:22.682892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b84e5-66b0-49c8-8a2e-727c9180d9da admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:22.687521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61a9d9-da03-4077-aeab-4d5d2d7722b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:22.688414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61a9d9-da03-4077-aeab-4d5d2d7722b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:22.688693 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61a9d9-da03-4077-aeab-4d5d2d7722b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:22.688968 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61a9d9-da03-4077-aeab-4d5d2d7722b6 admin admin] PATH_INFO: `/v3/projects/fdefda631db94e1bab383f0662ada08f/users/1e9a4ea1ba7a4c4fa4bcd50df1a81f7e/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:22.708617 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61a9d9-da03-4077-aeab-4d5d2d7722b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdefda631db94e1bab383f0662ada08f, user_id=1e9a4ea1ba7a4c4fa4bcd50df1a81f7e, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:22.711312 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61a9d9-da03-4077-aeab-4d5d2d7722b6 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:22.725374 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1283/2592] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:22 2022] PUT /identity/v3/projects/fdefda631db94e1bab383f0662ada08f/users/1e9a4ea1ba7a4c4fa4bcd50df1a81f7e/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:51:22.730016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83614cf2-e6ef-4a22-b42a-17d69113e8f8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:22.740500 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83614cf2-e6ef-4a22-b42a-17d69113e8f8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:22.801386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ae68d-8fb3-44f2-884f-2c832d7327c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:22.802093 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ae68d-8fb3-44f2-884f-2c832d7327c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:22.802381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ae68d-8fb3-44f2-884f-2c832d7327c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:22.802666 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ae68d-8fb3-44f2-884f-2c832d7327c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:22.803394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ae68d-8fb3-44f2-884f-2c832d7327c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:22.805012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ae68d-8fb3-44f2-884f-2c832d7327c8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:22.816332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1310/2593] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:22.821568 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61a9d9-da03-4077-aeab-4d5d2d7722b6 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:22.832791 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61a9d9-da03-4077-aeab-4d5d2d7722b6 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:22.837882 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec6c41-4dcc-4dff-aafd-626718b1cdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:22.838827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec6c41-4dcc-4dff-aafd-626718b1cdd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:22.839108 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec6c41-4dcc-4dff-aafd-626718b1cdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:22.839385 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec6c41-4dcc-4dff-aafd-626718b1cdd8 admin admin] PATH_INFO: `/v3/projects/fdefda631db94e1bab383f0662ada08f/users/1e9a4ea1ba7a4c4fa4bcd50df1a81f7e/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:22.842321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec6c41-4dcc-4dff-aafd-626718b1cdd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdefda631db94e1bab383f0662ada08f, user_id=1e9a4ea1ba7a4c4fa4bcd50df1a81f7e, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:22.846152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec6c41-4dcc-4dff-aafd-626718b1cdd8 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:22.859116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1284/2594] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:22 2022] PUT /identity/v3/projects/fdefda631db94e1bab383f0662ada08f/users/1e9a4ea1ba7a4c4fa4bcd50df1a81f7e/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:22.871058 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46fff8-d15e-46db-a321-330c866f5912 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:22.871592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46fff8-d15e-46db-a321-330c866f5912 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:22.872091 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46fff8-d15e-46db-a321-330c866f5912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:22.911250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9d46fff8-d15e-46db-a321-330c866f5912 None None] MFA Rules not processed for user `1e9a4ea1ba7a4c4fa4bcd50df1a81f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:22.912663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9d46fff8-d15e-46db-a321-330c866f5912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:22.950461 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:23.002965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1311/2595] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:23.014380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec6c41-4dcc-4dff-aafd-626718b1cdd8 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:23.016201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2fec6c41-4dcc-4dff-aafd-626718b1cdd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:23.075810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec6c41-4dcc-4dff-aafd-626718b1cdd8 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:23.079576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafcaee-21a5-4737-b1f2-3b6b5208981a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:23.080235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafcaee-21a5-4737-b1f2-3b6b5208981a service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:23.080628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafcaee-21a5-4737-b1f2-3b6b5208981a service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:23.080911 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafcaee-21a5-4737-b1f2-3b6b5208981a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:23.095296 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:23.144084 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafcaee-21a5-4737-b1f2-3b6b5208981a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:23.145575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafcaee-21a5-4737-b1f2-3b6b5208981a service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:23.155290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1285/2596] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:24.707281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46fff8-d15e-46db-a321-330c866f5912 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:24.709457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9d46fff8-d15e-46db-a321-330c866f5912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:24.723870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46fff8-d15e-46db-a321-330c866f5912 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:24.730166 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9230d-ff0d-45c5-ba41-c46ed0e7ceb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:24.731474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9230d-ff0d-45c5-ba41-c46ed0e7ceb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:24.731910 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9230d-ff0d-45c5-ba41-c46ed0e7ceb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:24.732354 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9230d-ff0d-45c5-ba41-c46ed0e7ceb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:24.809980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9230d-ff0d-45c5-ba41-c46ed0e7ceb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:24.811573 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9230d-ff0d-45c5-ba41-c46ed0e7ceb8 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:24.824086 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1312/2597] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:25.410126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafcaee-21a5-4737-b1f2-3b6b5208981a service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:25.419145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafcaee-21a5-4737-b1f2-3b6b5208981a service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:25.422810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6138f-c7b5-4edd-a345-2974e2d575fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:25.423478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6138f-c7b5-4edd-a345-2974e2d575fc service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:25.423758 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6138f-c7b5-4edd-a345-2974e2d575fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:25.424032 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6138f-c7b5-4edd-a345-2974e2d575fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:25.434596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6138f-c7b5-4edd-a345-2974e2d575fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:25.436870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6138f-c7b5-4edd-a345-2974e2d575fc service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:25.447660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1286/2598] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:29.345879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9230d-ff0d-45c5-ba41-c46ed0e7ceb8 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:29.409080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9230d-ff0d-45c5-ba41-c46ed0e7ceb8 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:29.413939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:29.414849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:29.415671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:29.415948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] PATH_INFO: `/v3/users/5b916df4be3d4ca1bb1f5166f7271824` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:29.417258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b916df4be3d4ca1bb1f5166f7271824)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:29.419243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:29.469438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b916df4be3d4ca1bb1f5166f7271824'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:29.478953 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b916df4be3d4ca1bb1f5166f7271824'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:29.479735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b916df4be3d4ca1bb1f5166f7271824'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:29.487076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b916df4be3d4ca1bb1f5166f7271824'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:29.487940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b916df4be3d4ca1bb1f5166f7271824'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:29.490061 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:29.510611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1313/2599] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:29 2022] DELETE /identity/v3/users/5b916df4be3d4ca1bb1f5166f7271824 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:51:29.515182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878dd3f-a5eb-4dcb-94c5-615e29d9ff1e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:29.515666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878dd3f-a5eb-4dcb-94c5-615e29d9ff1e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:29.515883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878dd3f-a5eb-4dcb-94c5-615e29d9ff1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:29.535460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-5878dd3f-a5eb-4dcb-94c5-615e29d9ff1e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:29.536656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5878dd3f-a5eb-4dcb-94c5-615e29d9ff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:29.600362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1287/2600] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:29.611051 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:29.612469 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:29.673924 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724456fb-ee37-4e3f-97cd-747a7aa217ed admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:29.677633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4eb5d0-89dc-4ad8-a45e-a3919a1c2c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:29.678331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eb5d0-89dc-4ad8-a45e-a3919a1c2c6e service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:29.678614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eb5d0-89dc-4ad8-a45e-a3919a1c2c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:29.678892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eb5d0-89dc-4ad8-a45e-a3919a1c2c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:29.688154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4eb5d0-89dc-4ad8-a45e-a3919a1c2c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:29.689590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4eb5d0-89dc-4ad8-a45e-a3919a1c2c6e service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:29.699617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1314/2601] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:29.967511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878dd3f-a5eb-4dcb-94c5-615e29d9ff1e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:29.968994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5878dd3f-a5eb-4dcb-94c5-615e29d9ff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:29.982337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878dd3f-a5eb-4dcb-94c5-615e29d9ff1e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:29.987437 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:29.988310 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:29.988624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:29.988899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] PATH_INFO: `/v3/projects/0de3d2e81d0a438bb2257d3f430e50d9` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:29.990141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de3d2e81d0a438bb2257d3f430e50d9)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:29.992072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:30.045199 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de3d2e81d0a438bb2257d3f430e50d9', 'request_id': 'req-c65f8158-7241-42b3-bff8-47de20ddf95d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.053503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de3d2e81d0a438bb2257d3f430e50d9', 'request_id': 'req-c65f8158-7241-42b3-bff8-47de20ddf95d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.062709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de3d2e81d0a438bb2257d3f430e50d9', 'request_id': 'req-c65f8158-7241-42b3-bff8-47de20ddf95d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.082860 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] The token cache is being invalidate because project 0de3d2e81d0a438bb2257d3f430e50d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:30.083323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.084298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.086767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1288/2602] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:29 2022] DELETE /identity/v3/projects/0de3d2e81d0a438bb2257d3f430e50d9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:30.091346 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4eb5d0-89dc-4ad8-a45e-a3919a1c2c6e service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:30.103228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4eb5d0-89dc-4ad8-a45e-a3919a1c2c6e service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:30.159107 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:30.159986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:30.160266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:30.160540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] PATH_INFO: `/v3/users/85320c1ed47545e0873f46a1cdc06d68` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:30.161810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] RBAC: Authorizing `identity:delete_user(user_id=85320c1ed47545e0873f46a1cdc06d68)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:30.163760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:30.211816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85320c1ed47545e0873f46a1cdc06d68'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.221085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85320c1ed47545e0873f46a1cdc06d68'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.222149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85320c1ed47545e0873f46a1cdc06d68'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.231054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85320c1ed47545e0873f46a1cdc06d68'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.231729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85320c1ed47545e0873f46a1cdc06d68'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.256649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1315/2603] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:30 2022] DELETE /identity/v3/users/85320c1ed47545e0873f46a1cdc06d68 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:30.549615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:30.551167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:30.564879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f8158-7241-42b3-bff8-47de20ddf95d admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:30.591477 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:30.627239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:30.628110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:30.628397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:30.628665 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] PATH_INFO: `/v3/projects/f7a3e29c257447f6b83c389643a0ed70` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:30.629857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a3e29c257447f6b83c389643a0ed70)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:30.631796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:30.679661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a3e29c257447f6b83c389643a0ed70', 'request_id': 'req-a8825a4e-4e19-4fd2-85b7-82a850f4d406'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.686621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a3e29c257447f6b83c389643a0ed70', 'request_id': 'req-a8825a4e-4e19-4fd2-85b7-82a850f4d406'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.692988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a3e29c257447f6b83c389643a0ed70', 'request_id': 'req-a8825a4e-4e19-4fd2-85b7-82a850f4d406'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.706013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] The token cache is being invalidate because project f7a3e29c257447f6b83c389643a0ed70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:30.706299 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.706912 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.708594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1289/2604] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:30 2022] DELETE /identity/v3/projects/f7a3e29c257447f6b83c389643a0ed70 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:30.712764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:30.714266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:30.722304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:30.728901 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970356f-cf70-4198-85ee-789ea44e979d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:30.788299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877b836-930a-4da6-92bc-d9531bed80ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:30.789204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877b836-930a-4da6-92bc-d9531bed80ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:30.789477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877b836-930a-4da6-92bc-d9531bed80ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:30.789754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877b836-930a-4da6-92bc-d9531bed80ad admin admin] PATH_INFO: `/v3/users/1e9a4ea1ba7a4c4fa4bcd50df1a81f7e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:30.791019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877b836-930a-4da6-92bc-d9531bed80ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e9a4ea1ba7a4c4fa4bcd50df1a81f7e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:30.792980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877b836-930a-4da6-92bc-d9531bed80ad admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:30.840570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f877b836-930a-4da6-92bc-d9531bed80ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e9a4ea1ba7a4c4fa4bcd50df1a81f7e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.850319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f877b836-930a-4da6-92bc-d9531bed80ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e9a4ea1ba7a4c4fa4bcd50df1a81f7e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.850941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f877b836-930a-4da6-92bc-d9531bed80ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e9a4ea1ba7a4c4fa4bcd50df1a81f7e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.858827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f877b836-930a-4da6-92bc-d9531bed80ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e9a4ea1ba7a4c4fa4bcd50df1a81f7e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.859483 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f877b836-930a-4da6-92bc-d9531bed80ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e9a4ea1ba7a4c4fa4bcd50df1a81f7e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:30.877016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1316/2605] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:30 2022] DELETE /identity/v3/users/1e9a4ea1ba7a4c4fa4bcd50df1a81f7e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:31.152817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:31.154412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:31.169476 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8825a4e-4e19-4fd2-85b7-82a850f4d406 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:31.230332 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa99481-b884-46a1-aa19-1fe67db9e228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:31.231119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa99481-b884-46a1-aa19-1fe67db9e228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:31.231401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa99481-b884-46a1-aa19-1fe67db9e228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:31.231675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa99481-b884-46a1-aa19-1fe67db9e228 admin admin] PATH_INFO: `/v3/projects/fdefda631db94e1bab383f0662ada08f` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:31.233123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa99481-b884-46a1-aa19-1fe67db9e228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdefda631db94e1bab383f0662ada08f)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:31.235072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa99481-b884-46a1-aa19-1fe67db9e228 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:31.282098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7fa99481-b884-46a1-aa19-1fe67db9e228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdefda631db94e1bab383f0662ada08f', 'request_id': 'req-7fa99481-b884-46a1-aa19-1fe67db9e228'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:31.288203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7fa99481-b884-46a1-aa19-1fe67db9e228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdefda631db94e1bab383f0662ada08f', 'request_id': 'req-7fa99481-b884-46a1-aa19-1fe67db9e228'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:31.292827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:31.295335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7fa99481-b884-46a1-aa19-1fe67db9e228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdefda631db94e1bab383f0662ada08f', 'request_id': 'req-7fa99481-b884-46a1-aa19-1fe67db9e228'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:31.308234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7fa99481-b884-46a1-aa19-1fe67db9e228 admin admin] The token cache is being invalidate because project fdefda631db94e1bab383f0662ada08f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:31.308544 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7fa99481-b884-46a1-aa19-1fe67db9e228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:31.309185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-7fa99481-b884-46a1-aa19-1fe67db9e228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:31.310846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1290/2606] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:31 2022] DELETE /identity/v3/projects/fdefda631db94e1bab383f0662ada08f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:31.315909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ea312-8ae6-40b6-a247-5644283550bf None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:31.316197 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ea312-8ae6-40b6-a247-5644283550bf None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:31.316472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ea312-8ae6-40b6-a247-5644283550bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:31.336519 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0b4ea312-8ae6-40b6-a247-5644283550bf None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:31.338314 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0b4ea312-8ae6-40b6-a247-5644283550bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:31.398027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1317/2607] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:31.406413 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d2f62-43d9-45b8-ac0e-2252d5eaf1c0 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:31.406705 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d2f62-43d9-45b8-ac0e-2252d5eaf1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:31.406986 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d2f62-43d9-45b8-ac0e-2252d5eaf1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:31.427500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-024d2f62-43d9-45b8-ac0e-2252d5eaf1c0 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:31.428717 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-024d2f62-43d9-45b8-ac0e-2252d5eaf1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:31.436898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1291/2608] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:31.441202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ea312-8ae6-40b6-a247-5644283550bf None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:31.453480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ea312-8ae6-40b6-a247-5644283550bf None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:31.458671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095f8ba-0457-4c07-b06b-e31625956e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:31.459339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095f8ba-0457-4c07-b06b-e31625956e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:31.459619 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095f8ba-0457-4c07-b06b-e31625956e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:31.459891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095f8ba-0457-4c07-b06b-e31625956e23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:31.460653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095f8ba-0457-4c07-b06b-e31625956e23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:51:31.460957 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095f8ba-0457-4c07-b06b-e31625956e23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:31.463257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095f8ba-0457-4c07-b06b-e31625956e23 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:31.476517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1318/2609] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:51:31.481586 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d2f62-43d9-45b8-ac0e-2252d5eaf1c0 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:31.491241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d2f62-43d9-45b8-ac0e-2252d5eaf1c0 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:31.495873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76e1eb-4aec-4a1d-a66e-31266a165c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:31.496529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76e1eb-4aec-4a1d-a66e-31266a165c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:31.496812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76e1eb-4aec-4a1d-a66e-31266a165c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:31.497096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76e1eb-4aec-4a1d-a66e-31266a165c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:31.497787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76e1eb-4aec-4a1d-a66e-31266a165c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:31.499739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76e1eb-4aec-4a1d-a66e-31266a165c30 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:31.527663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1292/2610] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:31 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:31.532078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095f8ba-0457-4c07-b06b-e31625956e23 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:31.543104 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095f8ba-0457-4c07-b06b-e31625956e23 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:31.547985 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:31.602071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4531d-9905-42a2-8c61-3b5c2d79e2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:31.603353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4531d-9905-42a2-8c61-3b5c2d79e2bd admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:31.603636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4531d-9905-42a2-8c61-3b5c2d79e2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:31.603914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4531d-9905-42a2-8c61-3b5c2d79e2bd admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:31.604606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4531d-9905-42a2-8c61-3b5c2d79e2bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:31.606579 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4531d-9905-42a2-8c61-3b5c2d79e2bd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:31.627564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1319/2611] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:31 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:31.632033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76e1eb-4aec-4a1d-a66e-31266a165c30 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:31.643115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76e1eb-4aec-4a1d-a66e-31266a165c30 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:31.647888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5bef0-a0ae-433c-b9e3-8831df149698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:31.648560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5bef0-a0ae-433c-b9e3-8831df149698 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:31.648847 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5bef0-a0ae-433c-b9e3-8831df149698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:31.649125 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5bef0-a0ae-433c-b9e3-8831df149698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:31.650509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5bef0-a0ae-433c-b9e3-8831df149698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:31.652145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5bef0-a0ae-433c-b9e3-8831df149698 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:31.662681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1293/2612] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:31.666837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4531d-9905-42a2-8c61-3b5c2d79e2bd admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:31.677069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4531d-9905-42a2-8c61-3b5c2d79e2bd admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:31.681880 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826df34-3a00-4e7a-855f-83829bea3276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:31.682567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826df34-3a00-4e7a-855f-83829bea3276 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:31.682853 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826df34-3a00-4e7a-855f-83829bea3276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:31.683132 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826df34-3a00-4e7a-855f-83829bea3276 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:31.683866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826df34-3a00-4e7a-855f-83829bea3276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:31.685478 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826df34-3a00-4e7a-855f-83829bea3276 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:31.697116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1320/2613] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:31.701156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5bef0-a0ae-433c-b9e3-8831df149698 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:31.711411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5bef0-a0ae-433c-b9e3-8831df149698 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:31.716827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db6c4c-8991-4ee4-b3d4-a641c918f0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:31.717834 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db6c4c-8991-4ee4-b3d4-a641c918f0e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:31.718044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db6c4c-8991-4ee4-b3d4-a641c918f0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:31.718327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db6c4c-8991-4ee4-b3d4-a641c918f0e8 admin admin] PATH_INFO: `/v3/projects/5572f729d7c7487c9619292573b90d70/users/a20445f40949478ca549f6df3d9dbd03/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:31.734042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db6c4c-8991-4ee4-b3d4-a641c918f0e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5572f729d7c7487c9619292573b90d70, user_id=a20445f40949478ca549f6df3d9dbd03, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:31.736644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db6c4c-8991-4ee4-b3d4-a641c918f0e8 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:31.755415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1294/2614] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:31 2022] PUT /identity/v3/projects/5572f729d7c7487c9619292573b90d70/users/a20445f40949478ca549f6df3d9dbd03/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:31.761245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1659656-7576-4bbe-b2f8-31fb46c9ae6f None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:31.761761 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1659656-7576-4bbe-b2f8-31fb46c9ae6f None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:31.761866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1659656-7576-4bbe-b2f8-31fb46c9ae6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:31.784960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a1659656-7576-4bbe-b2f8-31fb46c9ae6f None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:31.786233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a1659656-7576-4bbe-b2f8-31fb46c9ae6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:31.845848 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1321/2615] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:31.855534 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db6c4c-8991-4ee4-b3d4-a641c918f0e8 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:31.857234 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-42db6c4c-8991-4ee4-b3d4-a641c918f0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:31.919463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db6c4c-8991-4ee4-b3d4-a641c918f0e8 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:31.923164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e45f7-7799-4735-a3f7-51fa025f27b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:31.923827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e45f7-7799-4735-a3f7-51fa025f27b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:31.924110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e45f7-7799-4735-a3f7-51fa025f27b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:31.924388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e45f7-7799-4735-a3f7-51fa025f27b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:31.934019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e45f7-7799-4735-a3f7-51fa025f27b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:31.935464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e45f7-7799-4735-a3f7-51fa025f27b6 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:31.945435 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1295/2616] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:32.586262 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6541fa8-9d48-409f-8ccb-366221af6238 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:32.586617 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6541fa8-9d48-409f-8ccb-366221af6238 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:32.587649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6541fa8-9d48-409f-8ccb-366221af6238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:32.620562 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c6541fa8-9d48-409f-8ccb-366221af6238 None None] MFA Rules not processed for user `a20445f40949478ca549f6df3d9dbd03`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:32.621813 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c6541fa8-9d48-409f-8ccb-366221af6238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:32.687411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1322/2617] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:32.698708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e45f7-7799-4735-a3f7-51fa025f27b6 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:32.700130 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-816e45f7-7799-4735-a3f7-51fa025f27b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:32.770069 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e45f7-7799-4735-a3f7-51fa025f27b6 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:32.773773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb25319-15b2-4cf3-af19-a5dfa0994abe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:32.774500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb25319-15b2-4cf3-af19-a5dfa0994abe service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:32.774789 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb25319-15b2-4cf3-af19-a5dfa0994abe service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:32.775107 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb25319-15b2-4cf3-af19-a5dfa0994abe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:32.784486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb25319-15b2-4cf3-af19-a5dfa0994abe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:32.786055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb25319-15b2-4cf3-af19-a5dfa0994abe service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:32.795169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1296/2618] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:34.153628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f843694-db4e-4b6c-a505-b1c41effb20b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:34.154240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f843694-db4e-4b6c-a505-b1c41effb20b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:34.154240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f843694-db4e-4b6c-a505-b1c41effb20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:34.174518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2f843694-db4e-4b6c-a505-b1c41effb20b None None] MFA Rules not processed for user `a20445f40949478ca549f6df3d9dbd03`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:34.175871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2f843694-db4e-4b6c-a505-b1c41effb20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:34.183170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1323/2619] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:34.429830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb25319-15b2-4cf3-af19-a5dfa0994abe service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:34.439811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb25319-15b2-4cf3-af19-a5dfa0994abe service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:34.443428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d31686-b641-442d-aaa0-f49ed06deccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:34.444067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31686-b641-442d-aaa0-f49ed06deccd service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:34.444352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31686-b641-442d-aaa0-f49ed06deccd service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:34.444628 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31686-b641-442d-aaa0-f49ed06deccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:34.447068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d6d31686-b641-442d-aaa0-f49ed06deccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:34.456877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d31686-b641-442d-aaa0-f49ed06deccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:34.458341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d31686-b641-442d-aaa0-f49ed06deccd service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:34.466456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1297/2620] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:34.795351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f843694-db4e-4b6c-a505-b1c41effb20b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:34.796989 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2f843694-db4e-4b6c-a505-b1c41effb20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:34.818000 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:34.881143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f843694-db4e-4b6c-a505-b1c41effb20b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:34.886040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893c934-f8fb-4ed2-b7d6-a9f465c68856 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:34.886689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893c934-f8fb-4ed2-b7d6-a9f465c68856 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:34.886970 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893c934-f8fb-4ed2-b7d6-a9f465c68856 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:34.887244 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893c934-f8fb-4ed2-b7d6-a9f465c68856 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:34.889022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4893c934-f8fb-4ed2-b7d6-a9f465c68856 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:34.901003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893c934-f8fb-4ed2-b7d6-a9f465c68856 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:34.902647 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893c934-f8fb-4ed2-b7d6-a9f465c68856 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:34.913266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1324/2621] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:39.602618 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d31686-b641-442d-aaa0-f49ed06deccd service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:39.605538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-d6d31686-b641-442d-aaa0-f49ed06deccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:39.640319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:39.715197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d31686-b641-442d-aaa0-f49ed06deccd service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:39.720801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3e24c-878a-46b2-8985-0ad4da07de01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:39.721525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3e24c-878a-46b2-8985-0ad4da07de01 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:39.721810 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3e24c-878a-46b2-8985-0ad4da07de01 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:39.722144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3e24c-878a-46b2-8985-0ad4da07de01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:39.723999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-bbc3e24c-878a-46b2-8985-0ad4da07de01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:39.737274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3e24c-878a-46b2-8985-0ad4da07de01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:39.738971 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3e24c-878a-46b2-8985-0ad4da07de01 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:39.751721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1298/2622] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:51:44.908698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=27 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:49.746221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=27 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:51.985737 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893c934-f8fb-4ed2-b7d6-a9f465c68856 service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:51.987220 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4893c934-f8fb-4ed2-b7d6-a9f465c68856 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:52.003971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893c934-f8fb-4ed2-b7d6-a9f465c68856 service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:52.008586 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:52.009465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:52.009756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:52.010100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] PATH_INFO: `/v3/users/12486f89f982414da66f5546eaa5d4e6` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:52.011318 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12486f89f982414da66f5546eaa5d4e6)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:52.013260 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:52.069553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12486f89f982414da66f5546eaa5d4e6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.078458 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12486f89f982414da66f5546eaa5d4e6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.079296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12486f89f982414da66f5546eaa5d4e6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.086692 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12486f89f982414da66f5546eaa5d4e6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.087446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12486f89f982414da66f5546eaa5d4e6'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.107381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1325/2623] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:51 2022] DELETE /identity/v3/users/12486f89f982414da66f5546eaa5d4e6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:52.114169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4da990-16f5-4ced-b579-62d4927f66c3 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:52.114693 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4da990-16f5-4ced-b579-62d4927f66c3 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:52.115154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4da990-16f5-4ced-b579-62d4927f66c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:52.140577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-9d4da990-16f5-4ced-b579-62d4927f66c3 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:52.141799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9d4da990-16f5-4ced-b579-62d4927f66c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:52.208429 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1299/2624] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:52.218841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:52.220240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:52.278427 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445dc1f-765b-48c0-b335-3fd911362947 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:52.282076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1eea73-c682-4b98-864f-71df4b10a4e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:52.282732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1eea73-c682-4b98-864f-71df4b10a4e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:52.283016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1eea73-c682-4b98-864f-71df4b10a4e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:52.283290 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1eea73-c682-4b98-864f-71df4b10a4e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:52.293459 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1eea73-c682-4b98-864f-71df4b10a4e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:52.294939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1eea73-c682-4b98-864f-71df4b10a4e2 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:52.307673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1326/2625] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:51:52.584529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4da990-16f5-4ced-b579-62d4927f66c3 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:52.586158 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9d4da990-16f5-4ced-b579-62d4927f66c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:52.599346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4da990-16f5-4ced-b579-62d4927f66c3 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:52.604063 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:52.604997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:52.605275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:52.605547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] PATH_INFO: `/v3/projects/64070e0911364917ab2419538b531c1c` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:52.606774 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=64070e0911364917ab2419538b531c1c)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:52.608955 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:52.665301 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64070e0911364917ab2419538b531c1c', 'request_id': 'req-9499a962-9f13-4f7f-87bd-78ee0b304a0e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.672196 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64070e0911364917ab2419538b531c1c', 'request_id': 'req-9499a962-9f13-4f7f-87bd-78ee0b304a0e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.678045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64070e0911364917ab2419538b531c1c', 'request_id': 'req-9499a962-9f13-4f7f-87bd-78ee0b304a0e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.691237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] The token cache is being invalidate because project 64070e0911364917ab2419538b531c1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:52.691522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.692197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.693866 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1300/2626] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:52 2022] DELETE /identity/v3/projects/64070e0911364917ab2419538b531c1c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:52.697899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1eea73-c682-4b98-864f-71df4b10a4e2 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:52.708246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1eea73-c682-4b98-864f-71df4b10a4e2 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:52.764110 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05810607-4d03-4677-ad7a-9119c663425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:52.765011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05810607-4d03-4677-ad7a-9119c663425e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:52.765297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05810607-4d03-4677-ad7a-9119c663425e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:52.765577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05810607-4d03-4677-ad7a-9119c663425e admin admin] PATH_INFO: `/v3/users/d8a62e82a73f4dd98cc66b385979d3fa` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:52.766870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05810607-4d03-4677-ad7a-9119c663425e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8a62e82a73f4dd98cc66b385979d3fa)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:52.768816 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05810607-4d03-4677-ad7a-9119c663425e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:52.791803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:52.818138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-05810607-4d03-4677-ad7a-9119c663425e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8a62e82a73f4dd98cc66b385979d3fa'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.826879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-05810607-4d03-4677-ad7a-9119c663425e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8a62e82a73f4dd98cc66b385979d3fa'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.827608 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-05810607-4d03-4677-ad7a-9119c663425e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8a62e82a73f4dd98cc66b385979d3fa'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.835003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-05810607-4d03-4677-ad7a-9119c663425e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8a62e82a73f4dd98cc66b385979d3fa'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.835643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-05810607-4d03-4677-ad7a-9119c663425e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8a62e82a73f4dd98cc66b385979d3fa'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:52.853470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1327/2627] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:52 2022] DELETE /identity/v3/users/d8a62e82a73f4dd98cc66b385979d3fa => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:53.226462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:53.228087 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:53.253203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499a962-9f13-4f7f-87bd-78ee0b304a0e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:53.322560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b25cb-2dea-418f-a78f-7d66f0086f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:53.323442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b25cb-2dea-418f-a78f-7d66f0086f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:53.323853 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b25cb-2dea-418f-a78f-7d66f0086f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:53.323992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b25cb-2dea-418f-a78f-7d66f0086f55 admin admin] PATH_INFO: `/v3/projects/39eae4b98b314c9a81ea99693bd56095` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:53.325169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b25cb-2dea-418f-a78f-7d66f0086f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39eae4b98b314c9a81ea99693bd56095)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:53.327105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b25cb-2dea-418f-a78f-7d66f0086f55 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:53.359027 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:53.378058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-524b25cb-2dea-418f-a78f-7d66f0086f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39eae4b98b314c9a81ea99693bd56095', 'request_id': 'req-524b25cb-2dea-418f-a78f-7d66f0086f55'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:53.387120 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-524b25cb-2dea-418f-a78f-7d66f0086f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39eae4b98b314c9a81ea99693bd56095', 'request_id': 'req-524b25cb-2dea-418f-a78f-7d66f0086f55'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:53.395147 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-524b25cb-2dea-418f-a78f-7d66f0086f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39eae4b98b314c9a81ea99693bd56095', 'request_id': 'req-524b25cb-2dea-418f-a78f-7d66f0086f55'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:53.408934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-524b25cb-2dea-418f-a78f-7d66f0086f55 admin admin] The token cache is being invalidate because project 39eae4b98b314c9a81ea99693bd56095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:53.409223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-524b25cb-2dea-418f-a78f-7d66f0086f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:53.409849 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-524b25cb-2dea-418f-a78f-7d66f0086f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:53.411414 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1301/2628] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:53 2022] DELETE /identity/v3/projects/39eae4b98b314c9a81ea99693bd56095 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:53.418188 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6908e05-39d7-4926-85ec-ce5ced8e8cae None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:53.418934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6908e05-39d7-4926-85ec-ce5ced8e8cae None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:53.418934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6908e05-39d7-4926-85ec-ce5ced8e8cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:53.439400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-a6908e05-39d7-4926-85ec-ce5ced8e8cae None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:53.440669 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a6908e05-39d7-4926-85ec-ce5ced8e8cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:53.510212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1328/2629] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:53.519632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd304606-de4f-41ef-bbd9-83ce2dfda68e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:53.520003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd304606-de4f-41ef-bbd9-83ce2dfda68e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:53.520354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd304606-de4f-41ef-bbd9-83ce2dfda68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:53.540592 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-bd304606-de4f-41ef-bbd9-83ce2dfda68e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:53.541812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-bd304606-de4f-41ef-bbd9-83ce2dfda68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:53.550578 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1302/2630] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:51:53.554940 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6908e05-39d7-4926-85ec-ce5ced8e8cae None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:53.565701 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6908e05-39d7-4926-85ec-ce5ced8e8cae None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:53.570285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f10dca-2425-4fdd-ba4b-a7ad87478bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:53.570951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f10dca-2425-4fdd-ba4b-a7ad87478bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:53.571230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f10dca-2425-4fdd-ba4b-a7ad87478bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:53.571506 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f10dca-2425-4fdd-ba4b-a7ad87478bc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:53.572231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f10dca-2425-4fdd-ba4b-a7ad87478bc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:51:53.572534 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f10dca-2425-4fdd-ba4b-a7ad87478bc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:53.574316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f10dca-2425-4fdd-ba4b-a7ad87478bc6 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:53.591299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1329/2631] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:51:53.595651 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd304606-de4f-41ef-bbd9-83ce2dfda68e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:53.606315 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd304606-de4f-41ef-bbd9-83ce2dfda68e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:53.611353 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479e366-484d-441f-ae66-4377dd71c9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:53.612017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479e366-484d-441f-ae66-4377dd71c9eb admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:53.612300 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479e366-484d-441f-ae66-4377dd71c9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:53.612599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479e366-484d-441f-ae66-4377dd71c9eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:53.613309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479e366-484d-441f-ae66-4377dd71c9eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:53.615275 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479e366-484d-441f-ae66-4377dd71c9eb admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:53.644756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1303/2632] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:53 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:53.649154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f10dca-2425-4fdd-ba4b-a7ad87478bc6 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:53.659915 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f10dca-2425-4fdd-ba4b-a7ad87478bc6 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:53.718536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98c2ea-9275-4d9e-9fa9-7c114831e92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:53.719387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98c2ea-9275-4d9e-9fa9-7c114831e92f admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:53.719775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98c2ea-9275-4d9e-9fa9-7c114831e92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:53.720125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98c2ea-9275-4d9e-9fa9-7c114831e92f admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:53.720908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98c2ea-9275-4d9e-9fa9-7c114831e92f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:53.722964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98c2ea-9275-4d9e-9fa9-7c114831e92f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:53.746822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1330/2633] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:53 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:53.750914 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479e366-484d-441f-ae66-4377dd71c9eb admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:53.761558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479e366-484d-441f-ae66-4377dd71c9eb admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:53.766703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf348ed-883e-438e-9076-e8f3f008f3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:53.767454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf348ed-883e-438e-9076-e8f3f008f3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:53.767820 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf348ed-883e-438e-9076-e8f3f008f3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:53.768173 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf348ed-883e-438e-9076-e8f3f008f3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:53.768983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf348ed-883e-438e-9076-e8f3f008f3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:53.770961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf348ed-883e-438e-9076-e8f3f008f3f5 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:53.783664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1304/2634] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:53.789271 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98c2ea-9275-4d9e-9fa9-7c114831e92f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:53.800638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98c2ea-9275-4d9e-9fa9-7c114831e92f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:53.805422 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab4542-5b5f-4c20-ae3c-0a1896abc038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:53.806126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab4542-5b5f-4c20-ae3c-0a1896abc038 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:53.806413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab4542-5b5f-4c20-ae3c-0a1896abc038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:53.806695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab4542-5b5f-4c20-ae3c-0a1896abc038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:53.807427 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab4542-5b5f-4c20-ae3c-0a1896abc038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:53.809073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab4542-5b5f-4c20-ae3c-0a1896abc038 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:53.814511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:53.822077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1331/2635] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:53.826347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf348ed-883e-438e-9076-e8f3f008f3f5 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:53.837088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf348ed-883e-438e-9076-e8f3f008f3f5 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:53.842870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51b0de-288a-45fd-810e-5e02c04a04d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:53.843951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51b0de-288a-45fd-810e-5e02c04a04d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:53.844237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51b0de-288a-45fd-810e-5e02c04a04d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:53.844574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51b0de-288a-45fd-810e-5e02c04a04d7 admin admin] PATH_INFO: `/v3/projects/0ca29532a1644fb09eaaa2cbdf2f7934/users/536f0c95871d4b8cb5dba6726e4ba744/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:53.863585 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51b0de-288a-45fd-810e-5e02c04a04d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca29532a1644fb09eaaa2cbdf2f7934, user_id=536f0c95871d4b8cb5dba6726e4ba744, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:53.866335 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51b0de-288a-45fd-810e-5e02c04a04d7 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:53.881280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1305/2636] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:53 2022] PUT /identity/v3/projects/0ca29532a1644fb09eaaa2cbdf2f7934/users/536f0c95871d4b8cb5dba6726e4ba744/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:53.892415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9b8c6-d68d-4ab5-9cd2-c4664f29e115 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:53.893572 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9b8c6-d68d-4ab5-9cd2-c4664f29e115 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:53.893888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9b8c6-d68d-4ab5-9cd2-c4664f29e115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:53.926179 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-84a9b8c6-d68d-4ab5-9cd2-c4664f29e115 None None] MFA Rules not processed for user `536f0c95871d4b8cb5dba6726e4ba744`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:53.927549 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-84a9b8c6-d68d-4ab5-9cd2-c4664f29e115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:53.999003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1332/2637] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:54.003642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51b0de-288a-45fd-810e-5e02c04a04d7 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:54.067618 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51b0de-288a-45fd-810e-5e02c04a04d7 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:54.072354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf19388-8710-4a75-9dae-201307b1b1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:54.073013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf19388-8710-4a75-9dae-201307b1b1cf admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.073292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf19388-8710-4a75-9dae-201307b1b1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.074112 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf19388-8710-4a75-9dae-201307b1b1cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.074813 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf19388-8710-4a75-9dae-201307b1b1cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:54.076722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf19388-8710-4a75-9dae-201307b1b1cf admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:54.112678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1306/2638] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:54 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:54.114307 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9b8c6-d68d-4ab5-9cd2-c4664f29e115 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:54.125121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9b8c6-d68d-4ab5-9cd2-c4664f29e115 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:54.181934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afaaf06-bbd2-47fc-a978-91e92007cc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:54.182637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaaf06-bbd2-47fc-a978-91e92007cc5a admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.182923 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaaf06-bbd2-47fc-a978-91e92007cc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.183202 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaaf06-bbd2-47fc-a978-91e92007cc5a admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.183914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afaaf06-bbd2-47fc-a978-91e92007cc5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:54.185886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afaaf06-bbd2-47fc-a978-91e92007cc5a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:54.207258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1333/2639] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:54 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:54.211279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf19388-8710-4a75-9dae-201307b1b1cf admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:54.222133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf19388-8710-4a75-9dae-201307b1b1cf admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:54.226736 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fedd58-c05e-4c29-85ed-674ec6ff1f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:54.227389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fedd58-c05e-4c29-85ed-674ec6ff1f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.227669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fedd58-c05e-4c29-85ed-674ec6ff1f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.227943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fedd58-c05e-4c29-85ed-674ec6ff1f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.228672 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fedd58-c05e-4c29-85ed-674ec6ff1f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:54.230311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fedd58-c05e-4c29-85ed-674ec6ff1f11 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:54.242062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1307/2640] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:54.246323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afaaf06-bbd2-47fc-a978-91e92007cc5a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:54.257147 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afaaf06-bbd2-47fc-a978-91e92007cc5a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:54.262369 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca8cb0-97ab-416c-8132-5df021d4a75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:54.263035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca8cb0-97ab-416c-8132-5df021d4a75e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.263342 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca8cb0-97ab-416c-8132-5df021d4a75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.263618 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca8cb0-97ab-416c-8132-5df021d4a75e admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.264345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca8cb0-97ab-416c-8132-5df021d4a75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:54.265960 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca8cb0-97ab-416c-8132-5df021d4a75e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:54.277175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1334/2641] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:54.282488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fedd58-c05e-4c29-85ed-674ec6ff1f11 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:54.293495 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fedd58-c05e-4c29-85ed-674ec6ff1f11 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:54.298397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fbed0-ecd1-4122-b1a2-f172552ade9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:54.299336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fbed0-ecd1-4122-b1a2-f172552ade9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.299621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fbed0-ecd1-4122-b1a2-f172552ade9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.299898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fbed0-ecd1-4122-b1a2-f172552ade9b admin admin] PATH_INFO: `/v3/projects/d6f868e4ba0e485b923930d1e0d118c3/users/332c15d13a51464dbd30e74fa91e1232/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.315467 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fbed0-ecd1-4122-b1a2-f172552ade9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f868e4ba0e485b923930d1e0d118c3, user_id=332c15d13a51464dbd30e74fa91e1232, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:54.318688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fbed0-ecd1-4122-b1a2-f172552ade9b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:54.331060 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:54.336877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1308/2642] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:54 2022] PUT /identity/v3/projects/d6f868e4ba0e485b923930d1e0d118c3/users/332c15d13a51464dbd30e74fa91e1232/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:54.348829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a595a0-dff8-4d14-aaa2-06bdcdfc9ad2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.349117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a595a0-dff8-4d14-aaa2-06bdcdfc9ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.349394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a595a0-dff8-4d14-aaa2-06bdcdfc9ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.387046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d7a595a0-dff8-4d14-aaa2-06bdcdfc9ad2 None None] MFA Rules not processed for user `332c15d13a51464dbd30e74fa91e1232`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:54.388285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d7a595a0-dff8-4d14-aaa2-06bdcdfc9ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:54.448236 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:54.459810 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1335/2643] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:54.460885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fbed0-ecd1-4122-b1a2-f172552ade9b admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:54.524587 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fbed0-ecd1-4122-b1a2-f172552ade9b admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:54.529290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3d42a-ed3d-4310-9119-88116def4c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:54.529957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3d42a-ed3d-4310-9119-88116def4c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.530247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3d42a-ed3d-4310-9119-88116def4c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.530520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3d42a-ed3d-4310-9119-88116def4c76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.531221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3d42a-ed3d-4310-9119-88116def4c76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:54.533110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3d42a-ed3d-4310-9119-88116def4c76 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:54.566056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1309/2644] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:51:54 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:51:54.570249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a595a0-dff8-4d14-aaa2-06bdcdfc9ad2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:54.581518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a595a0-dff8-4d14-aaa2-06bdcdfc9ad2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:54.635980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b998ecb-8028-41e7-aed8-991562bdecd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:54.637152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b998ecb-8028-41e7-aed8-991562bdecd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.637431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b998ecb-8028-41e7-aed8-991562bdecd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.637704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b998ecb-8028-41e7-aed8-991562bdecd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.638413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b998ecb-8028-41e7-aed8-991562bdecd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:54.640330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b998ecb-8028-41e7-aed8-991562bdecd7 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:54.661213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1336/2645] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:51:54 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:51:54.666277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3d42a-ed3d-4310-9119-88116def4c76 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:54.677606 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3d42a-ed3d-4310-9119-88116def4c76 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:54.682262 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ca9b7-c853-49a2-aeee-5a9e8fa9a2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:54.683446 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ca9b7-c853-49a2-aeee-5a9e8fa9a2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.683753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ca9b7-c853-49a2-aeee-5a9e8fa9a2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.684024 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ca9b7-c853-49a2-aeee-5a9e8fa9a2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.684745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ca9b7-c853-49a2-aeee-5a9e8fa9a2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:54.686358 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ca9b7-c853-49a2-aeee-5a9e8fa9a2c0 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:54.696823 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1310/2646] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:54.701649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b998ecb-8028-41e7-aed8-991562bdecd7 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:54.711762 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b998ecb-8028-41e7-aed8-991562bdecd7 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:54.716978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9c9e4-0e84-4870-bcae-5bcde951d699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:54.717858 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9c9e4-0e84-4870-bcae-5bcde951d699 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.718182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9c9e4-0e84-4870-bcae-5bcde951d699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.718460 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9c9e4-0e84-4870-bcae-5bcde951d699 admin admin] PATH_INFO: `/v3/projects/a47adf8a66f541189a2fba3fe5c99ca7/users/3054906c21bd49128aff8feeca85205d/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.733591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9c9e4-0e84-4870-bcae-5bcde951d699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47adf8a66f541189a2fba3fe5c99ca7, user_id=3054906c21bd49128aff8feeca85205d, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:54.736299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9c9e4-0e84-4870-bcae-5bcde951d699 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:54.749292 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1337/2647] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:54 2022] PUT /identity/v3/projects/a47adf8a66f541189a2fba3fe5c99ca7/users/3054906c21bd49128aff8feeca85205d/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:54.753321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ca9b7-c853-49a2-aeee-5a9e8fa9a2c0 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:54.763503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ca9b7-c853-49a2-aeee-5a9e8fa9a2c0 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:54.818749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e14932-910e-4135-a60a-d1831e2ad4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:54.819408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e14932-910e-4135-a60a-d1831e2ad4c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.819692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e14932-910e-4135-a60a-d1831e2ad4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.819967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e14932-910e-4135-a60a-d1831e2ad4c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.820734 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e14932-910e-4135-a60a-d1831e2ad4c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:54.822293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e14932-910e-4135-a60a-d1831e2ad4c9 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:54.832486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1311/2648] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:51:54.836677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9c9e4-0e84-4870-bcae-5bcde951d699 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:54.847865 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9c9e4-0e84-4870-bcae-5bcde951d699 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:54.852543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885644f-ec70-463a-9e1f-0055953c90b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:54.853431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885644f-ec70-463a-9e1f-0055953c90b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.853724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885644f-ec70-463a-9e1f-0055953c90b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.854117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885644f-ec70-463a-9e1f-0055953c90b4 admin admin] PATH_INFO: `/v3/projects/a47adf8a66f541189a2fba3fe5c99ca7/users/3054906c21bd49128aff8feeca85205d/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.856187 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885644f-ec70-463a-9e1f-0055953c90b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47adf8a66f541189a2fba3fe5c99ca7, user_id=3054906c21bd49128aff8feeca85205d, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:54.859143 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885644f-ec70-463a-9e1f-0055953c90b4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:54.869374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1338/2649] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:51:54 2022] PUT /identity/v3/projects/a47adf8a66f541189a2fba3fe5c99ca7/users/3054906c21bd49128aff8feeca85205d/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:54.880550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7073cd3-a5ce-4743-b883-4e014ce4eacb None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:54.880838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7073cd3-a5ce-4743-b883-4e014ce4eacb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:54.881116 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7073cd3-a5ce-4743-b883-4e014ce4eacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:54.909176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-e7073cd3-a5ce-4743-b883-4e014ce4eacb None None] MFA Rules not processed for user `3054906c21bd49128aff8feeca85205d`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:54.910464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e7073cd3-a5ce-4743-b883-4e014ce4eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:54.969560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:54.980913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1312/2650] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:54.992074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885644f-ec70-463a-9e1f-0055953c90b4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:54.993780 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f885644f-ec70-463a-9e1f-0055953c90b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:55.052200 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885644f-ec70-463a-9e1f-0055953c90b4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:55.055876 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efb21c-d735-44af-b124-f2340cc3303a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:55.056516 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efb21c-d735-44af-b124-f2340cc3303a service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:55.056798 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efb21c-d735-44af-b124-f2340cc3303a service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:55.057077 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efb21c-d735-44af-b124-f2340cc3303a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:55.066583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efb21c-d735-44af-b124-f2340cc3303a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:55.068013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efb21c-d735-44af-b124-f2340cc3303a service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:55.077411 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1339/2651] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:55.118240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7073cd3-a5ce-4743-b883-4e014ce4eacb None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:55.119608 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e7073cd3-a5ce-4743-b883-4e014ce4eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:55.132291 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7073cd3-a5ce-4743-b883-4e014ce4eacb None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:55.135877 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706da9c0-f062-4877-8d5e-2f25abfead73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:55.136515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706da9c0-f062-4877-8d5e-2f25abfead73 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:55.136792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706da9c0-f062-4877-8d5e-2f25abfead73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:55.137064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706da9c0-f062-4877-8d5e-2f25abfead73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:55.193024 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706da9c0-f062-4877-8d5e-2f25abfead73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:55.194500 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706da9c0-f062-4877-8d5e-2f25abfead73 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:55.203401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1313/2652] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:55.452843 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7974c-ed81-4a30-a146-be17a0e204b9 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:55.453429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7974c-ed81-4a30-a146-be17a0e204b9 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:55.453429 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7974c-ed81-4a30-a146-be17a0e204b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:55.485764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-eda7974c-ed81-4a30-a146-be17a0e204b9 None None] MFA Rules not processed for user `a20445f40949478ca549f6df3d9dbd03`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:55.487031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-eda7974c-ed81-4a30-a146-be17a0e204b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:55.553751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:55.564180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1340/2653] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:58.130059 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706da9c0-f062-4877-8d5e-2f25abfead73 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:58.210462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706da9c0-f062-4877-8d5e-2f25abfead73 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:58.215685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:58.216610 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:58.216897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:58.217174 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] PATH_INFO: `/v3/users/536f0c95871d4b8cb5dba6726e4ba744` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:58.218454 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=536f0c95871d4b8cb5dba6726e4ba744)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:58.220384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:58.272320 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '536f0c95871d4b8cb5dba6726e4ba744'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:58.273137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '536f0c95871d4b8cb5dba6726e4ba744'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:58.273884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '536f0c95871d4b8cb5dba6726e4ba744'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:58.281890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '536f0c95871d4b8cb5dba6726e4ba744'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:58.293630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '536f0c95871d4b8cb5dba6726e4ba744'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:58.326185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1314/2654] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:58 2022] DELETE /identity/v3/users/536f0c95871d4b8cb5dba6726e4ba744 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:58.330870 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282d0d-6fd6-422a-9bea-ac06af1faea0 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:58.331652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282d0d-6fd6-422a-9bea-ac06af1faea0 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:58.331652 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282d0d-6fd6-422a-9bea-ac06af1faea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:58.352991 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-4d282d0d-6fd6-422a-9bea-ac06af1faea0 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:51:58.354308 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4d282d0d-6fd6-422a-9bea-ac06af1faea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:58.426895 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1341/2655] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:51:58.445804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:58.448523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:58.473997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:58.529775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c48ed5-22ec-4423-9901-4fdb8addfff5 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:58.533546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f174e-ad7f-4654-b9a2-8d07955e66be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:58.534309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f174e-ad7f-4654-b9a2-8d07955e66be service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:58.534666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f174e-ad7f-4654-b9a2-8d07955e66be service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:58.535017 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f174e-ad7f-4654-b9a2-8d07955e66be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:58.546565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f174e-ad7f-4654-b9a2-8d07955e66be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:58.548039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f174e-ad7f-4654-b9a2-8d07955e66be service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:58.558708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1315/2656] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:51:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:51:59.026723 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282d0d-6fd6-422a-9bea-ac06af1faea0 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:59.028138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4d282d0d-6fd6-422a-9bea-ac06af1faea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:59.041439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282d0d-6fd6-422a-9bea-ac06af1faea0 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:59.046691 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:59.047689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:59.048078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:59.048448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] PATH_INFO: `/v3/projects/0ca29532a1644fb09eaaa2cbdf2f7934` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:59.049817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca29532a1644fb09eaaa2cbdf2f7934)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:59.052295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:59.102024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca29532a1644fb09eaaa2cbdf2f7934', 'request_id': 'req-a15394f1-8296-48e2-bca8-12a512ff0ea9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.110854 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca29532a1644fb09eaaa2cbdf2f7934', 'request_id': 'req-a15394f1-8296-48e2-bca8-12a512ff0ea9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.117580 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca29532a1644fb09eaaa2cbdf2f7934', 'request_id': 'req-a15394f1-8296-48e2-bca8-12a512ff0ea9'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.128367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] The token cache is being invalidate because project 0ca29532a1644fb09eaaa2cbdf2f7934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:59.128655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.129284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.130952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1342/2657] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:59 2022] DELETE /identity/v3/projects/0ca29532a1644fb09eaaa2cbdf2f7934 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:59.135043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f174e-ad7f-4654-b9a2-8d07955e66be service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:59.145410 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f174e-ad7f-4654-b9a2-8d07955e66be service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:59.206566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:59.207447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:59.207729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:59.208003 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] PATH_INFO: `/v3/users/332c15d13a51464dbd30e74fa91e1232` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:59.209190 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=332c15d13a51464dbd30e74fa91e1232)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:59.211164 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:59.260269 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '332c15d13a51464dbd30e74fa91e1232'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.261549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '332c15d13a51464dbd30e74fa91e1232'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.262197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '332c15d13a51464dbd30e74fa91e1232'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.269109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '332c15d13a51464dbd30e74fa91e1232'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.275910 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '332c15d13a51464dbd30e74fa91e1232'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.293407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1316/2658] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:59 2022] DELETE /identity/v3/users/332c15d13a51464dbd30e74fa91e1232 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:51:59.558387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:59.560786 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:59.577371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15394f1-8296-48e2-bca8-12a512ff0ea9 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:59.638212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:59.639098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:59.639384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:59.639660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] PATH_INFO: `/v3/projects/d6f868e4ba0e485b923930d1e0d118c3` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:59.640887 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f868e4ba0e485b923930d1e0d118c3)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:59.642933 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:59.669089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:59.694559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f868e4ba0e485b923930d1e0d118c3', 'request_id': 'req-609ae101-6819-4bc6-bc2c-519b12b069e1'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.702470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f868e4ba0e485b923930d1e0d118c3', 'request_id': 'req-609ae101-6819-4bc6-bc2c-519b12b069e1'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.708714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f868e4ba0e485b923930d1e0d118c3', 'request_id': 'req-609ae101-6819-4bc6-bc2c-519b12b069e1'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.719655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] The token cache is being invalidate because project d6f868e4ba0e485b923930d1e0d118c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:51:59.719942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.720576 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.722162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1343/2659] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:51:59 2022] DELETE /identity/v3/projects/d6f868e4ba0e485b923930d1e0d118c3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:51:59.726119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:51:59.727611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:51:59.741045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4aee1a-2f3e-4b3b-ae81-a8c854dca920 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:51:59.772930 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:51:59.799643 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfcd02-46b2-415f-926f-628fd6edfadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:51:59.800635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfcd02-46b2-415f-926f-628fd6edfadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:51:59.800925 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfcd02-46b2-415f-926f-628fd6edfadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:51:59.801201 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfcd02-46b2-415f-926f-628fd6edfadc admin admin] PATH_INFO: `/v3/users/3054906c21bd49128aff8feeca85205d` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:51:59.802416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfcd02-46b2-415f-926f-628fd6edfadc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3054906c21bd49128aff8feeca85205d)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:51:59.804342 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfcd02-46b2-415f-926f-628fd6edfadc admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:51:59.849075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-44cfcd02-46b2-415f-926f-628fd6edfadc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3054906c21bd49128aff8feeca85205d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.849709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-44cfcd02-46b2-415f-926f-628fd6edfadc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3054906c21bd49128aff8feeca85205d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.850331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-44cfcd02-46b2-415f-926f-628fd6edfadc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3054906c21bd49128aff8feeca85205d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.857767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-44cfcd02-46b2-415f-926f-628fd6edfadc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3054906c21bd49128aff8feeca85205d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.865355 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-44cfcd02-46b2-415f-926f-628fd6edfadc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3054906c21bd49128aff8feeca85205d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:51:59.882701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1317/2660] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:51:59 2022] DELETE /identity/v3/users/3054906c21bd49128aff8feeca85205d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:00.149043 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:00.150510 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:00.171231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ae101-6819-4bc6-bc2c-519b12b069e1 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:00.231939 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca1987-b06b-49e0-9605-5e08a8a10c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:00.232807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca1987-b06b-49e0-9605-5e08a8a10c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:00.233085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca1987-b06b-49e0-9605-5e08a8a10c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:00.233613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca1987-b06b-49e0-9605-5e08a8a10c06 admin admin] PATH_INFO: `/v3/projects/a47adf8a66f541189a2fba3fe5c99ca7` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:00.234827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca1987-b06b-49e0-9605-5e08a8a10c06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a47adf8a66f541189a2fba3fe5c99ca7)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:00.236729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca1987-b06b-49e0-9605-5e08a8a10c06 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:00.283470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ecca1987-b06b-49e0-9605-5e08a8a10c06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a47adf8a66f541189a2fba3fe5c99ca7', 'request_id': 'req-ecca1987-b06b-49e0-9605-5e08a8a10c06'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:00.290909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ecca1987-b06b-49e0-9605-5e08a8a10c06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a47adf8a66f541189a2fba3fe5c99ca7', 'request_id': 'req-ecca1987-b06b-49e0-9605-5e08a8a10c06'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:00.297099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ecca1987-b06b-49e0-9605-5e08a8a10c06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a47adf8a66f541189a2fba3fe5c99ca7', 'request_id': 'req-ecca1987-b06b-49e0-9605-5e08a8a10c06'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:00.307564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ecca1987-b06b-49e0-9605-5e08a8a10c06 admin admin] The token cache is being invalidate because project a47adf8a66f541189a2fba3fe5c99ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:52:00.307855 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ecca1987-b06b-49e0-9605-5e08a8a10c06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:00.308518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-ecca1987-b06b-49e0-9605-5e08a8a10c06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:00.310089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1344/2661] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:52:00 2022] DELETE /identity/v3/projects/a47adf8a66f541189a2fba3fe5c99ca7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:00.316024 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b331b3-64d4-4c29-90db-b4bb063a985a None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:00.316518 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b331b3-64d4-4c29-90db-b4bb063a985a None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:00.316619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b331b3-64d4-4c29-90db-b4bb063a985a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:00.336072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-43b331b3-64d4-4c29-90db-b4bb063a985a None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:00.337277 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-43b331b3-64d4-4c29-90db-b4bb063a985a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:00.395249 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1318/2662] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:00.404383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3431c3d-9dff-43cf-9cb4-ef982bf05ddb None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:00.404606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3431c3d-9dff-43cf-9cb4-ef982bf05ddb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:00.404879 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3431c3d-9dff-43cf-9cb4-ef982bf05ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:00.424128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c3431c3d-9dff-43cf-9cb4-ef982bf05ddb None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:00.425391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c3431c3d-9dff-43cf-9cb4-ef982bf05ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:00.433721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1345/2663] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:00.437897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b331b3-64d4-4c29-90db-b4bb063a985a None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:00.447581 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b331b3-64d4-4c29-90db-b4bb063a985a None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:00.452956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdfb2b-ffb5-4be4-ab41-1700e83c63d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:00.453612 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdfb2b-ffb5-4be4-ab41-1700e83c63d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:00.453889 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdfb2b-ffb5-4be4-ab41-1700e83c63d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:00.454188 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdfb2b-ffb5-4be4-ab41-1700e83c63d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:00.454917 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdfb2b-ffb5-4be4-ab41-1700e83c63d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:52:00.455221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdfb2b-ffb5-4be4-ab41-1700e83c63d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:00.456843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdfb2b-ffb5-4be4-ab41-1700e83c63d9 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:00.468257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1319/2664] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:52:00.472583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3431c3d-9dff-43cf-9cb4-ef982bf05ddb None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:00.481811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3431c3d-9dff-43cf-9cb4-ef982bf05ddb None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:00.486725 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18df7f-e8a2-448c-8a81-40bd111fcd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:00.487370 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18df7f-e8a2-448c-8a81-40bd111fcd31 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:00.487689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18df7f-e8a2-448c-8a81-40bd111fcd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:00.487965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18df7f-e8a2-448c-8a81-40bd111fcd31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:00.488658 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18df7f-e8a2-448c-8a81-40bd111fcd31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:00.490598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18df7f-e8a2-448c-8a81-40bd111fcd31 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:00.519024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1346/2665] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:52:00 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:52:00.521972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdfb2b-ffb5-4be4-ab41-1700e83c63d9 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:00.532030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdfb2b-ffb5-4be4-ab41-1700e83c63d9 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:00.578814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:00.596872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550633c-27f8-42c3-8e80-edb009672b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:00.597660 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550633c-27f8-42c3-8e80-edb009672b3e admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:00.599200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550633c-27f8-42c3-8e80-edb009672b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:00.599480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550633c-27f8-42c3-8e80-edb009672b3e admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:00.600182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550633c-27f8-42c3-8e80-edb009672b3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:00.603203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550633c-27f8-42c3-8e80-edb009672b3e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:00.627744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1320/2666] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:52:00 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:52:00.631915 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18df7f-e8a2-448c-8a81-40bd111fcd31 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:00.643150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18df7f-e8a2-448c-8a81-40bd111fcd31 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:00.647803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee12807-32ff-4712-ad8a-89a337b2e15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:00.648601 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee12807-32ff-4712-ad8a-89a337b2e15d admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:00.648868 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee12807-32ff-4712-ad8a-89a337b2e15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:00.649141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee12807-32ff-4712-ad8a-89a337b2e15d admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:00.649894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee12807-32ff-4712-ad8a-89a337b2e15d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:00.651522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee12807-32ff-4712-ad8a-89a337b2e15d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:00.661961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1347/2667] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:52:00.665857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550633c-27f8-42c3-8e80-edb009672b3e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:00.675899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550633c-27f8-42c3-8e80-edb009672b3e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:00.680484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f0146-4f6e-4b15-9911-a2d17c2cb428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:00.681138 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f0146-4f6e-4b15-9911-a2d17c2cb428 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:00.681416 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f0146-4f6e-4b15-9911-a2d17c2cb428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:00.681687 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f0146-4f6e-4b15-9911-a2d17c2cb428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:00.682474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f0146-4f6e-4b15-9911-a2d17c2cb428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:00.684598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f0146-4f6e-4b15-9911-a2d17c2cb428 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:00.696075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1321/2668] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:52:00.700078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee12807-32ff-4712-ad8a-89a337b2e15d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:00.705419 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:00.712250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee12807-32ff-4712-ad8a-89a337b2e15d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:00.718537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4a35-85e4-4ffd-b90e-dbda2760028d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:00.719439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4a35-85e4-4ffd-b90e-dbda2760028d admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:00.719718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4a35-85e4-4ffd-b90e-dbda2760028d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:00.719990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4a35-85e4-4ffd-b90e-dbda2760028d admin admin] PATH_INFO: `/v3/projects/1db3898787d649df9ac3648410f0b6d7/users/063f718df1e04f7cbc78d21a114ebc85/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:00.735834 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a4a35-85e4-4ffd-b90e-dbda2760028d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1db3898787d649df9ac3648410f0b6d7, user_id=063f718df1e04f7cbc78d21a114ebc85, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:00.738558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a4a35-85e4-4ffd-b90e-dbda2760028d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:00.753449 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1348/2669] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:52:00 2022] PUT /identity/v3/projects/1db3898787d649df9ac3648410f0b6d7/users/063f718df1e04f7cbc78d21a114ebc85/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:00.765214 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a99c4-b34c-47cd-9da7-5898b8e469ca None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:00.765649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a99c4-b34c-47cd-9da7-5898b8e469ca None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:00.765760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a99c4-b34c-47cd-9da7-5898b8e469ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:00.794462 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-334a99c4-b34c-47cd-9da7-5898b8e469ca None None] MFA Rules not processed for user `063f718df1e04f7cbc78d21a114ebc85`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:00.795676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-334a99c4-b34c-47cd-9da7-5898b8e469ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:00.867171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1322/2670] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:00.872115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4a35-85e4-4ffd-b90e-dbda2760028d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:00.934569 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4a35-85e4-4ffd-b90e-dbda2760028d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:00.939156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e90ae1-5c3e-438a-99d2-85a9d9748aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:00.939817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e90ae1-5c3e-438a-99d2-85a9d9748aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:00.940101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e90ae1-5c3e-438a-99d2-85a9d9748aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:00.940379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e90ae1-5c3e-438a-99d2-85a9d9748aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:00.941073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e90ae1-5c3e-438a-99d2-85a9d9748aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:00.943015 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e90ae1-5c3e-438a-99d2-85a9d9748aa0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:00.968745 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1349/2671] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:52:00 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:52:00.972896 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a99c4-b34c-47cd-9da7-5898b8e469ca None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:00.984261 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a99c4-b34c-47cd-9da7-5898b8e469ca None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:01.040311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447ba28-118e-4f7d-98a8-46a230ffe721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:01.040984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447ba28-118e-4f7d-98a8-46a230ffe721 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:01.041268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447ba28-118e-4f7d-98a8-46a230ffe721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:01.041543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447ba28-118e-4f7d-98a8-46a230ffe721 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:01.042803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447ba28-118e-4f7d-98a8-46a230ffe721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:01.044723 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447ba28-118e-4f7d-98a8-46a230ffe721 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:01.068503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1323/2672] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:52:00 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:52:01.072288 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e90ae1-5c3e-438a-99d2-85a9d9748aa0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:01.082607 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e90ae1-5c3e-438a-99d2-85a9d9748aa0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:01.088164 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca99b1-7a21-407b-b180-76422ee7ebfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:01.088818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca99b1-7a21-407b-b180-76422ee7ebfd admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:01.089101 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca99b1-7a21-407b-b180-76422ee7ebfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:01.089380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca99b1-7a21-407b-b180-76422ee7ebfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:01.090262 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca99b1-7a21-407b-b180-76422ee7ebfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:01.091896 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca99b1-7a21-407b-b180-76422ee7ebfd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:01.103677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1350/2673] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:52:01.108280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447ba28-118e-4f7d-98a8-46a230ffe721 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:01.118461 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447ba28-118e-4f7d-98a8-46a230ffe721 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:01.123699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4411d-7c82-493b-acb0-5e9ed384a193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:01.124573 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4411d-7c82-493b-acb0-5e9ed384a193 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:01.124852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4411d-7c82-493b-acb0-5e9ed384a193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:01.125124 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4411d-7c82-493b-acb0-5e9ed384a193 admin admin] PATH_INFO: `/v3/projects/1e80334b6ea94890807e899b36077b5e/users/87933a6d18a54e77816242c244c9a7c2/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:01.139954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4411d-7c82-493b-acb0-5e9ed384a193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e80334b6ea94890807e899b36077b5e, user_id=87933a6d18a54e77816242c244c9a7c2, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:01.142730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4411d-7c82-493b-acb0-5e9ed384a193 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:01.155646 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1324/2674] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:52:01 2022] PUT /identity/v3/projects/1e80334b6ea94890807e899b36077b5e/users/87933a6d18a54e77816242c244c9a7c2/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:01.159807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca99b1-7a21-407b-b180-76422ee7ebfd admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:01.170518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca99b1-7a21-407b-b180-76422ee7ebfd admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:01.241098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a5aae-7569-4604-891d-e271d43b90e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:01.241885 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a5aae-7569-4604-891d-e271d43b90e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:01.242300 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a5aae-7569-4604-891d-e271d43b90e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:01.242573 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a5aae-7569-4604-891d-e271d43b90e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:01.243303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a5aae-7569-4604-891d-e271d43b90e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:01.244908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a5aae-7569-4604-891d-e271d43b90e9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:01.258703 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1351/2675] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:52:01.259830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4411d-7c82-493b-acb0-5e9ed384a193 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:01.270533 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4411d-7c82-493b-acb0-5e9ed384a193 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:01.275243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cbe5c-1a4a-40f6-839c-e9c2a7e6eb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:01.276128 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cbe5c-1a4a-40f6-839c-e9c2a7e6eb4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:01.276978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cbe5c-1a4a-40f6-839c-e9c2a7e6eb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:01.277256 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cbe5c-1a4a-40f6-839c-e9c2a7e6eb4b admin admin] PATH_INFO: `/v3/projects/1e80334b6ea94890807e899b36077b5e/users/87933a6d18a54e77816242c244c9a7c2/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:01.279295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cbe5c-1a4a-40f6-839c-e9c2a7e6eb4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e80334b6ea94890807e899b36077b5e, user_id=87933a6d18a54e77816242c244c9a7c2, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:01.281964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cbe5c-1a4a-40f6-839c-e9c2a7e6eb4b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:01.292742 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1325/2676] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:52:01 2022] PUT /identity/v3/projects/1e80334b6ea94890807e899b36077b5e/users/87933a6d18a54e77816242c244c9a7c2/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:52:01.361279 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9e555-25e5-4624-ab89-6ec795348474 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:01.361572 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9e555-25e5-4624-ab89-6ec795348474 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:01.361838 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9e555-25e5-4624-ab89-6ec795348474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:01.391732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-21d9e555-25e5-4624-ab89-6ec795348474 None None] MFA Rules not processed for user `87933a6d18a54e77816242c244c9a7c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:01.393007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-21d9e555-25e5-4624-ab89-6ec795348474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:01.433707 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:01.470115 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1352/2677] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:01.477379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cbe5c-1a4a-40f6-839c-e9c2a7e6eb4b admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:01.479221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f88cbe5c-1a4a-40f6-839c-e9c2a7e6eb4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:01.485778 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:01.543347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cbe5c-1a4a-40f6-839c-e9c2a7e6eb4b admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:01.547106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a900feb-fffe-4d8f-930e-924969d8f5d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:01.547770 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a900feb-fffe-4d8f-930e-924969d8f5d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:01.548056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a900feb-fffe-4d8f-930e-924969d8f5d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:01.548333 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a900feb-fffe-4d8f-930e-924969d8f5d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:01.747510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a900feb-fffe-4d8f-930e-924969d8f5d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:01.748861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a900feb-fffe-4d8f-930e-924969d8f5d6 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:01.757901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1326/2678] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:52:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:52:04.619586 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9e555-25e5-4624-ab89-6ec795348474 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:04.621273 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-21d9e555-25e5-4624-ab89-6ec795348474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:04.720965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9e555-25e5-4624-ab89-6ec795348474 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:04.730119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b05dcb-0325-445e-a718-1db5c1541aff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:04.731151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b05dcb-0325-445e-a718-1db5c1541aff service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:04.731437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b05dcb-0325-445e-a718-1db5c1541aff service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:04.731715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b05dcb-0325-445e-a718-1db5c1541aff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:04.734799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-37b05dcb-0325-445e-a718-1db5c1541aff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:04.748775 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b05dcb-0325-445e-a718-1db5c1541aff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:04.751036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b05dcb-0325-445e-a718-1db5c1541aff service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:04.763634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1353/2679] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:52:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:52:06.074780 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a900feb-fffe-4d8f-930e-924969d8f5d6 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:06.076200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5a900feb-fffe-4d8f-930e-924969d8f5d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:06.087424 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a900feb-fffe-4d8f-930e-924969d8f5d6 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:06.091039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7b198-d94d-4f62-84d6-44c382024f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:06.091677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7b198-d94d-4f62-84d6-44c382024f5d service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:06.091958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7b198-d94d-4f62-84d6-44c382024f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:06.092232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7b198-d94d-4f62-84d6-44c382024f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:06.103320 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7b198-d94d-4f62-84d6-44c382024f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:06.105253 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7b198-d94d-4f62-84d6-44c382024f5d service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:06.114981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1327/2680] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:52:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:52:11.025032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b05dcb-0325-445e-a718-1db5c1541aff service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:11.099982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b05dcb-0325-445e-a718-1db5c1541aff service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:11.105195 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:11.106088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:11.106376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:11.106653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] PATH_INFO: `/v3/users/063f718df1e04f7cbc78d21a114ebc85` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:11.109095 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] RBAC: Authorizing `identity:delete_user(user_id=063f718df1e04f7cbc78d21a114ebc85)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:11.111075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:11.183208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '063f718df1e04f7cbc78d21a114ebc85'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.193173 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '063f718df1e04f7cbc78d21a114ebc85'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.194111 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '063f718df1e04f7cbc78d21a114ebc85'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.201299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:11.203621 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '063f718df1e04f7cbc78d21a114ebc85'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.204841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '063f718df1e04f7cbc78d21a114ebc85'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.225004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1354/2681] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:52:11 2022] DELETE /identity/v3/users/063f718df1e04f7cbc78d21a114ebc85 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:11.229794 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd5934-4e0d-4959-a935-82c7cf66c99d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:11.230325 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd5934-4e0d-4959-a935-82c7cf66c99d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:11.230420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd5934-4e0d-4959-a935-82c7cf66c99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:11.250031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-acdd5934-4e0d-4959-a935-82c7cf66c99d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:11.251235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-acdd5934-4e0d-4959-a935-82c7cf66c99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:11.331032 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1328/2682] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:11.340739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:11.342196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:11.401781 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411cd22-47b5-4812-9563-cdc4fa07080e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:11.405803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1c35a-b04a-4060-939d-49acddf67b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:11.406492 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1c35a-b04a-4060-939d-49acddf67b5b service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:11.406817 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1c35a-b04a-4060-939d-49acddf67b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:11.407081 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1c35a-b04a-4060-939d-49acddf67b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:11.416684 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1c35a-b04a-4060-939d-49acddf67b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:11.418694 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1c35a-b04a-4060-939d-49acddf67b5b service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:11.428454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1355/2683] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:52:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:52:11.715384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd5934-4e0d-4959-a935-82c7cf66c99d None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:11.716983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-acdd5934-4e0d-4959-a935-82c7cf66c99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:11.731058 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd5934-4e0d-4959-a935-82c7cf66c99d None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:11.736676 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:11.737800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:11.738230 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:11.738624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] PATH_INFO: `/v3/projects/1db3898787d649df9ac3648410f0b6d7` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:11.740100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1db3898787d649df9ac3648410f0b6d7)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:11.742395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:11.783268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:11.790850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1db3898787d649df9ac3648410f0b6d7', 'request_id': 'req-0638af78-1ae2-4bdc-8d68-e7780176f3bc'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.797515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1db3898787d649df9ac3648410f0b6d7', 'request_id': 'req-0638af78-1ae2-4bdc-8d68-e7780176f3bc'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.802932 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1db3898787d649df9ac3648410f0b6d7', 'request_id': 'req-0638af78-1ae2-4bdc-8d68-e7780176f3bc'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.816878 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] The token cache is being invalidate because project 1db3898787d649df9ac3648410f0b6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:52:11.817371 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.818255 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.819816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1329/2684] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:52:11 2022] DELETE /identity/v3/projects/1db3898787d649df9ac3648410f0b6d7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:11.823959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1c35a-b04a-4060-939d-49acddf67b5b service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:11.834075 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1c35a-b04a-4060-939d-49acddf67b5b service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:11.890566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a99cc5-7ca9-49f2-ae6d-fa0ca935df44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:11.891440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a99cc5-7ca9-49f2-ae6d-fa0ca935df44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:11.891721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a99cc5-7ca9-49f2-ae6d-fa0ca935df44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:11.891998 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a99cc5-7ca9-49f2-ae6d-fa0ca935df44 admin admin] PATH_INFO: `/v3/users/87933a6d18a54e77816242c244c9a7c2` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:11.893177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a99cc5-7ca9-49f2-ae6d-fa0ca935df44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87933a6d18a54e77816242c244c9a7c2)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:11.895145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a99cc5-7ca9-49f2-ae6d-fa0ca935df44 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:11.941080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f3a99cc5-7ca9-49f2-ae6d-fa0ca935df44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87933a6d18a54e77816242c244c9a7c2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.948873 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f3a99cc5-7ca9-49f2-ae6d-fa0ca935df44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87933a6d18a54e77816242c244c9a7c2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.949739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f3a99cc5-7ca9-49f2-ae6d-fa0ca935df44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87933a6d18a54e77816242c244c9a7c2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.957303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f3a99cc5-7ca9-49f2-ae6d-fa0ca935df44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87933a6d18a54e77816242c244c9a7c2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.957989 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f3a99cc5-7ca9-49f2-ae6d-fa0ca935df44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87933a6d18a54e77816242c244c9a7c2'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:11.972004 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:11.977507 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1356/2685] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:52:11 2022] DELETE /identity/v3/users/87933a6d18a54e77816242c244c9a7c2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:12.278788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:12.278788 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:12.290052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638af78-1ae2-4bdc-8d68-e7780176f3bc admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:12.382575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe42c5-76ed-413d-ab37-74ea88f9e734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:12.383366 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe42c5-76ed-413d-ab37-74ea88f9e734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:12.383644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe42c5-76ed-413d-ab37-74ea88f9e734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:12.383915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe42c5-76ed-413d-ab37-74ea88f9e734 admin admin] PATH_INFO: `/v3/projects/1e80334b6ea94890807e899b36077b5e` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:12.385098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe42c5-76ed-413d-ab37-74ea88f9e734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e80334b6ea94890807e899b36077b5e)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:12.387969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe42c5-76ed-413d-ab37-74ea88f9e734 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:12.444512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f3fe42c5-76ed-413d-ab37-74ea88f9e734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e80334b6ea94890807e899b36077b5e', 'request_id': 'req-f3fe42c5-76ed-413d-ab37-74ea88f9e734'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:12.451816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f3fe42c5-76ed-413d-ab37-74ea88f9e734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e80334b6ea94890807e899b36077b5e', 'request_id': 'req-f3fe42c5-76ed-413d-ab37-74ea88f9e734'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:12.458436 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f3fe42c5-76ed-413d-ab37-74ea88f9e734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e80334b6ea94890807e899b36077b5e', 'request_id': 'req-f3fe42c5-76ed-413d-ab37-74ea88f9e734'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:12.472669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f3fe42c5-76ed-413d-ab37-74ea88f9e734 admin admin] The token cache is being invalidate because project 1e80334b6ea94890807e899b36077b5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:52:12.472959 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f3fe42c5-76ed-413d-ab37-74ea88f9e734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:12.473602 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-f3fe42c5-76ed-413d-ab37-74ea88f9e734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:12.475970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1330/2686] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:52:12 2022] DELETE /identity/v3/projects/1e80334b6ea94890807e899b36077b5e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:52:12.483176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a422a-ce6f-4499-82fe-62abac1e96a1 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:12.483997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a422a-ce6f-4499-82fe-62abac1e96a1 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:12.484319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a422a-ce6f-4499-82fe-62abac1e96a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:12.504809 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-296a422a-ce6f-4499-82fe-62abac1e96a1 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:12.506234 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-296a422a-ce6f-4499-82fe-62abac1e96a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:12.576608 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1357/2687] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:12.586923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb96c3-6194-43a2-9a47-8b0afea08b35 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:12.587200 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb96c3-6194-43a2-9a47-8b0afea08b35 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:12.587464 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb96c3-6194-43a2-9a47-8b0afea08b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:12.609621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-6dfb96c3-6194-43a2-9a47-8b0afea08b35 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:12.610949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6dfb96c3-6194-43a2-9a47-8b0afea08b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:12.621619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1331/2688] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:12.625948 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a422a-ce6f-4499-82fe-62abac1e96a1 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:12.636232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a422a-ce6f-4499-82fe-62abac1e96a1 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:12.640861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04504731-908e-46c2-a02f-d90b4f592f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:12.641471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04504731-908e-46c2-a02f-d90b4f592f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:12.641852 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04504731-908e-46c2-a02f-d90b4f592f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:12.642031 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04504731-908e-46c2-a02f-d90b4f592f57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:12.642783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04504731-908e-46c2-a02f-d90b4f592f57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:52:12.643085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04504731-908e-46c2-a02f-d90b4f592f57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:12.644814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04504731-908e-46c2-a02f-d90b4f592f57 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:12.661654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1358/2689] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:52:12.662980 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb96c3-6194-43a2-9a47-8b0afea08b35 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:12.674111 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb96c3-6194-43a2-9a47-8b0afea08b35 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:12.679236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccd2ff-2ecc-4bab-a298-83b49d76ec4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:12.679990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccd2ff-2ecc-4bab-a298-83b49d76ec4e admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:12.680290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccd2ff-2ecc-4bab-a298-83b49d76ec4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:12.680606 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccd2ff-2ecc-4bab-a298-83b49d76ec4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:12.681339 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccd2ff-2ecc-4bab-a298-83b49d76ec4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:12.683363 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccd2ff-2ecc-4bab-a298-83b49d76ec4e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:12.716589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1332/2690] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:52:12 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:52:12.727356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04504731-908e-46c2-a02f-d90b4f592f57 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:12.747997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04504731-908e-46c2-a02f-d90b4f592f57 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:12.811912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7025897-49fb-4142-8e45-6b1aae39d6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:12.812598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7025897-49fb-4142-8e45-6b1aae39d6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:12.812882 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7025897-49fb-4142-8e45-6b1aae39d6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:12.813168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7025897-49fb-4142-8e45-6b1aae39d6f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:12.813857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7025897-49fb-4142-8e45-6b1aae39d6f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:12.816515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7025897-49fb-4142-8e45-6b1aae39d6f3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:12.843470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1359/2691] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:52:12 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:52:12.847719 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccd2ff-2ecc-4bab-a298-83b49d76ec4e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:12.860941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccd2ff-2ecc-4bab-a298-83b49d76ec4e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:12.866208 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c562843-7ed3-4f92-9ddd-a5d93cc1b3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:12.866961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c562843-7ed3-4f92-9ddd-a5d93cc1b3b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:12.867228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c562843-7ed3-4f92-9ddd-a5d93cc1b3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:12.867508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c562843-7ed3-4f92-9ddd-a5d93cc1b3b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:12.868203 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c562843-7ed3-4f92-9ddd-a5d93cc1b3b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:12.870192 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c562843-7ed3-4f92-9ddd-a5d93cc1b3b6 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:12.880655 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1333/2692] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:52:12.884944 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7025897-49fb-4142-8e45-6b1aae39d6f3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:12.896084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7025897-49fb-4142-8e45-6b1aae39d6f3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:12.909047 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a01a0d-09d2-402c-853a-dd0ae327247c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:12.909839 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a01a0d-09d2-402c-853a-dd0ae327247c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:12.910146 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a01a0d-09d2-402c-853a-dd0ae327247c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:12.910511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a01a0d-09d2-402c-853a-dd0ae327247c admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:12.911384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a01a0d-09d2-402c-853a-dd0ae327247c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:12.913063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a01a0d-09d2-402c-853a-dd0ae327247c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:12.924511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1360/2693] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:52:12.929383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c562843-7ed3-4f92-9ddd-a5d93cc1b3b6 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:12.941695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c562843-7ed3-4f92-9ddd-a5d93cc1b3b6 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:12.948021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fdaa2-49dd-4277-895b-481902f53912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:12.950885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fdaa2-49dd-4277-895b-481902f53912 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:12.951357 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fdaa2-49dd-4277-895b-481902f53912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:12.952031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fdaa2-49dd-4277-895b-481902f53912 admin admin] PATH_INFO: `/v3/projects/b678cb6fa6b340ae9cfc877a240ab213/users/b953dd2dc515426683501b4207839548/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:12.972961 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fdaa2-49dd-4277-895b-481902f53912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b678cb6fa6b340ae9cfc877a240ab213, user_id=b953dd2dc515426683501b4207839548, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:12.975632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fdaa2-49dd-4277-895b-481902f53912 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:12.978180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:12.994178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1334/2694] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:52:12 2022] PUT /identity/v3/projects/b678cb6fa6b340ae9cfc877a240ab213/users/b953dd2dc515426683501b4207839548/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:52:13.006988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c4cc0-e78f-466f-a720-23f054856547 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:13.010394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c4cc0-e78f-466f-a720-23f054856547 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:13.010394 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c4cc0-e78f-466f-a720-23f054856547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:13.040267 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-546c4cc0-e78f-466f-a720-23f054856547 None None] MFA Rules not processed for user `b953dd2dc515426683501b4207839548`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:13.041501 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-546c4cc0-e78f-466f-a720-23f054856547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:13.098795 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:13.112371 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1361/2695] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:13.123664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fdaa2-49dd-4277-895b-481902f53912 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:13.125397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-752fdaa2-49dd-4277-895b-481902f53912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:13.194463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fdaa2-49dd-4277-895b-481902f53912 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:13.198281 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9670c10-87d1-4c99-96ae-0d3732bd5af5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:13.199523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9670c10-87d1-4c99-96ae-0d3732bd5af5 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:13.199808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9670c10-87d1-4c99-96ae-0d3732bd5af5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:13.200086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9670c10-87d1-4c99-96ae-0d3732bd5af5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:13.213274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9670c10-87d1-4c99-96ae-0d3732bd5af5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:13.214797 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9670c10-87d1-4c99-96ae-0d3732bd5af5 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:13.225109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1335/2696] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:52:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:52:18.287933 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264a299-6329-4308-a906-610f4ef555df None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:18.288438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264a299-6329-4308-a906-610f4ef555df None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:18.288553 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264a299-6329-4308-a906-610f4ef555df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:18.340311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-d264a299-6329-4308-a906-610f4ef555df None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:18.341583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d264a299-6329-4308-a906-610f4ef555df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:18.410097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1362/2697] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:52:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:18.463681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9670c10-87d1-4c99-96ae-0d3732bd5af5 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:18.465162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c9670c10-87d1-4c99-96ae-0d3732bd5af5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:18.558433 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9670c10-87d1-4c99-96ae-0d3732bd5af5 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:18.563532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058df65-c420-4026-a4db-b04e9c399963 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:18.564213 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058df65-c420-4026-a4db-b04e9c399963 service swift] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:18.564488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058df65-c420-4026-a4db-b04e9c399963 service swift] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:18.564798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058df65-c420-4026-a4db-b04e9c399963 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:18.575900 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058df65-c420-4026-a4db-b04e9c399963 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:18.578577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058df65-c420-4026-a4db-b04e9c399963 service swift] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:18.587812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1336/2698] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:52:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:52:18.883323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999781d-f49b-4d85-af0b-89ac75c7e8ef None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:18.884302 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999781d-f49b-4d85-af0b-89ac75c7e8ef None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:18.885599 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999781d-f49b-4d85-af0b-89ac75c7e8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:18.918687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-0999781d-f49b-4d85-af0b-89ac75c7e8ef None None] MFA Rules not processed for user `b953dd2dc515426683501b4207839548`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:18.920306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0999781d-f49b-4d85-af0b-89ac75c7e8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:18.934710 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1363/2699] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:28.585489 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:INSERT=1,keystone:SELECT=69 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:28.913684 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=45 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:34.747992 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe902f2-eca4-4da6-b5ff-3402dce42796 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:34.748547 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe902f2-eca4-4da6-b5ff-3402dce42796 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:34.748861 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe902f2-eca4-4da6-b5ff-3402dce42796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:34.770771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-9fe902f2-eca4-4da6-b5ff-3402dce42796 None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:34.772069 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9fe902f2-eca4-4da6-b5ff-3402dce42796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:34.779792 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1337/2700] 10.222.0.60 () {58 vars in 967 bytes} [Mon Jul 25 12:52:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:34.792734 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999781d-f49b-4d85-af0b-89ac75c7e8ef None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:34.809291 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999781d-f49b-4d85-af0b-89ac75c7e8ef None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:34.816163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137c98c-6b65-49ec-b43f-1eabddf62cb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:34.816825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137c98c-6b65-49ec-b43f-1eabddf62cb4 service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:34.817109 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137c98c-6b65-49ec-b43f-1eabddf62cb4 service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:34.817388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137c98c-6b65-49ec-b43f-1eabddf62cb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:34.836364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137c98c-6b65-49ec-b43f-1eabddf62cb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:34.836364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137c98c-6b65-49ec-b43f-1eabddf62cb4 service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:34.839749 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1364/2701] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:52:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:52:36.172692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cafb4-4240-4397-968a-7f77cc8ef231 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:36.173220 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cafb4-4240-4397-968a-7f77cc8ef231 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:36.173220 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cafb4-4240-4397-968a-7f77cc8ef231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:36.193527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-a66cafb4-4240-4397-968a-7f77cc8ef231 None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:36.194774 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a66cafb4-4240-4397-968a-7f77cc8ef231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:36.201948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1338/2702] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:52:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:36.214162 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137c98c-6b65-49ec-b43f-1eabddf62cb4 service swift] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:36.225205 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137c98c-6b65-49ec-b43f-1eabddf62cb4 service swift] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:36.231934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502c243-89e0-4873-8856-80288bc965a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:36.232582 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502c243-89e0-4873-8856-80288bc965a3 service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:36.232863 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502c243-89e0-4873-8856-80288bc965a3 service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:36.233135 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502c243-89e0-4873-8856-80288bc965a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:36.242346 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502c243-89e0-4873-8856-80288bc965a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:36.244066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502c243-89e0-4873-8856-80288bc965a3 service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:36.254559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1365/2703] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:52:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:52:36.826488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e83f0-85cd-4f39-9d94-1612e7edf715 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:36.826924 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e83f0-85cd-4f39-9d94-1612e7edf715 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:36.827272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e83f0-85cd-4f39-9d94-1612e7edf715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:36.847206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-7a6e83f0-85cd-4f39-9d94-1612e7edf715 None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:36.848530 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7a6e83f0-85cd-4f39-9d94-1612e7edf715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:36.857064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1339/2704] 10.222.0.60 () {58 vars in 967 bytes} [Mon Jul 25 12:52:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:36.870881 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502c243-89e0-4873-8856-80288bc965a3 service swift] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:36.899058 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502c243-89e0-4873-8856-80288bc965a3 service swift] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:36.905183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a103c0-3081-48c2-b5e5-b0f8fba00bd0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:36.905518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a103c0-3081-48c2-b5e5-b0f8fba00bd0 service swift] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:36.905718 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a103c0-3081-48c2-b5e5-b0f8fba00bd0 service swift] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:36.905894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a103c0-3081-48c2-b5e5-b0f8fba00bd0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:36.920690 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a103c0-3081-48c2-b5e5-b0f8fba00bd0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:36.923170 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a103c0-3081-48c2-b5e5-b0f8fba00bd0 service swift] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:36.934351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1366/2705] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:52:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:52:36.988979 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e83f0-85cd-4f39-9d94-1612e7edf715 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:36.990642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7a6e83f0-85cd-4f39-9d94-1612e7edf715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:37.053634 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a103c0-3081-48c2-b5e5-b0f8fba00bd0 service swift] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:37.055744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-04a103c0-3081-48c2-b5e5-b0f8fba00bd0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:37.106875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e83f0-85cd-4f39-9d94-1612e7edf715 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:37.115453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab22477-0749-4b25-b8ce-a82aa669ca42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:37.116129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab22477-0749-4b25-b8ce-a82aa669ca42 service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:37.116408 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab22477-0749-4b25-b8ce-a82aa669ca42 service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:37.116683 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab22477-0749-4b25-b8ce-a82aa669ca42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:37.118934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5ab22477-0749-4b25-b8ce-a82aa669ca42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:37.133388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab22477-0749-4b25-b8ce-a82aa669ca42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:37.135100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab22477-0749-4b25-b8ce-a82aa669ca42 service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:37.149680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1340/2706] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:52:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:52:37.155451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a103c0-3081-48c2-b5e5-b0f8fba00bd0 service swift] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:37.160270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:37.161259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:37.161624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:37.162012 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] PATH_INFO: `/v3/users/a20445f40949478ca549f6df3d9dbd03` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:37.163343 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20445f40949478ca549f6df3d9dbd03)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:37.165496 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:37.229640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cb7fd-4687-496d-bb4b-0f93f758a2da None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:37.230240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cb7fd-4687-496d-bb4b-0f93f758a2da None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:37.230557 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cb7fd-4687-496d-bb4b-0f93f758a2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:37.236821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20445f40949478ca549f6df3d9dbd03'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:37.247005 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20445f40949478ca549f6df3d9dbd03'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:37.247837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20445f40949478ca549f6df3d9dbd03'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:37.253292 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-268cb7fd-4687-496d-bb4b-0f93f758a2da None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:37.254542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-268cb7fd-4687-496d-bb4b-0f93f758a2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:37.262206 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20445f40949478ca549f6df3d9dbd03'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:37.263211 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20445f40949478ca549f6df3d9dbd03'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:37.264845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1341/2707] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Jul 25 12:52:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:52:37.283720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cb7fd-4687-496d-bb4b-0f93f758a2da None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:37.285367 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-268cb7fd-4687-496d-bb4b-0f93f758a2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:37.287899 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1367/2708] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:52:37 2022] DELETE /identity/v3/users/a20445f40949478ca549f6df3d9dbd03 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:37.300863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cb7fd-4687-496d-bb4b-0f93f758a2da None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:37.366681 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29f1e8-9f89-4c57-b790-48e32bf52389 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:37.367352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29f1e8-9f89-4c57-b790-48e32bf52389 service swift] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:37.367640 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29f1e8-9f89-4c57-b790-48e32bf52389 service swift] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:37.367916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29f1e8-9f89-4c57-b790-48e32bf52389 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:37.444964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29f1e8-9f89-4c57-b790-48e32bf52389 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:37.446747 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29f1e8-9f89-4c57-b790-48e32bf52389 service swift] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:37.456885 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1342/2709] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:52:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 12:52:37.488360 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:37.491286 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:37.566084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:37.604927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d75e36-d844-4749-b19f-d1ce8f1394ef admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:37.610715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:37.611594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:37.611876 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:37.612366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] PATH_INFO: `/v3/projects/5572f729d7c7487c9619292573b90d70` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:37.617537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5572f729d7c7487c9619292573b90d70)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:37.617537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:37.704134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5572f729d7c7487c9619292573b90d70', 'request_id': 'req-bdf9c048-8df1-4be2-ad1b-e53458cc4325'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:37.714902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5572f729d7c7487c9619292573b90d70', 'request_id': 'req-bdf9c048-8df1-4be2-ad1b-e53458cc4325'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:37.729313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5572f729d7c7487c9619292573b90d70', 'request_id': 'req-bdf9c048-8df1-4be2-ad1b-e53458cc4325'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:37.744412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] The token cache is being invalidate because project 5572f729d7c7487c9619292573b90d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:52:37.751886 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:37.754424 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:37.756598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1368/2710] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:52:37 2022] DELETE /identity/v3/projects/5572f729d7c7487c9619292573b90d70 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:37.761950 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac8368-7343-4c83-a9d9-854f68b0760d None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:37.762388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac8368-7343-4c83-a9d9-854f68b0760d None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:37.762675 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac8368-7343-4c83-a9d9-854f68b0760d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:37.775381 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:37.790594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-b3ac8368-7343-4c83-a9d9-854f68b0760d None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:37.792104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-b3ac8368-7343-4c83-a9d9-854f68b0760d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:37.826690 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:37.836277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:37.889884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1343/2711] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:37.891632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec557abe-78e6-40ab-a150-7e4d2a26de9b None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:37.892085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec557abe-78e6-40ab-a150-7e4d2a26de9b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:37.892537 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec557abe-78e6-40ab-a150-7e4d2a26de9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:37.914739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-ec557abe-78e6-40ab-a150-7e4d2a26de9b None None] MFA Rules not processed for user `6428e810f10c434ba32d073a542b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:37.915978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ec557abe-78e6-40ab-a150-7e4d2a26de9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:37.932214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9c048-8df1-4be2-ad1b-e53458cc4325 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:37.936761 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f978be-fa03-4a06-9fbc-91585255f09e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:37.937491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f978be-fa03-4a06-9fbc-91585255f09e service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:37.937862 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f978be-fa03-4a06-9fbc-91585255f09e service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:37.938238 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f978be-fa03-4a06-9fbc-91585255f09e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:37.940374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f6f978be-fa03-4a06-9fbc-91585255f09e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:37.986173 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1344/2712] 10.222.0.60 () {58 vars in 967 bytes} [Mon Jul 25 12:52:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:37.987915 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342f7b9-2a54-457d-b75a-3c1290bf4ace None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:37.988499 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342f7b9-2a54-457d-b75a-3c1290bf4ace None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:37.989139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342f7b9-2a54-457d-b75a-3c1290bf4ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:38.015776 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-a342f7b9-2a54-457d-b75a-3c1290bf4ace None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:38.016999 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a342f7b9-2a54-457d-b75a-3c1290bf4ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:38.020189 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f978be-fa03-4a06-9fbc-91585255f09e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:38.021712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f978be-fa03-4a06-9fbc-91585255f09e service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:38.031678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1345/2713] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:38.034229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342f7b9-2a54-457d-b75a-3c1290bf4ace None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:38.035613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a342f7b9-2a54-457d-b75a-3c1290bf4ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:38.036230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1369/2714] 10.222.0.60 () {60 vars in 1396 bytes} [Mon Jul 25 12:52:37 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3242 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:52:38.036950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f978be-fa03-4a06-9fbc-91585255f09e service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:38.052438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f978be-fa03-4a06-9fbc-91585255f09e service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:38.068903 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85167d25-13cf-422e-bf7a-4aadad58f2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:38.069539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85167d25-13cf-422e-bf7a-4aadad58f2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:38.069830 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85167d25-13cf-422e-bf7a-4aadad58f2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:38.070194 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85167d25-13cf-422e-bf7a-4aadad58f2e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:38.070943 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85167d25-13cf-422e-bf7a-4aadad58f2e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:52:38.071255 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85167d25-13cf-422e-bf7a-4aadad58f2e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:38.073008 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85167d25-13cf-422e-bf7a-4aadad58f2e9 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:38.090347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1370/2715] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:52:38.092521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85167d25-13cf-422e-bf7a-4aadad58f2e9 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:38.105474 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85167d25-13cf-422e-bf7a-4aadad58f2e9 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:38.110590 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a171e0b-c522-4fb8-8a32-0528e847e784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:38.111268 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a171e0b-c522-4fb8-8a32-0528e847e784 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:38.111558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a171e0b-c522-4fb8-8a32-0528e847e784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:38.111841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a171e0b-c522-4fb8-8a32-0528e847e784 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:38.112557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a171e0b-c522-4fb8-8a32-0528e847e784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:38.114544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a171e0b-c522-4fb8-8a32-0528e847e784 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:38.126050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:38.144730 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342f7b9-2a54-457d-b75a-3c1290bf4ace None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:38.151103 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0e591-c7a7-4956-9dcb-6f47eb82887d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56b0d48e08c4f778a7c509c10edc871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:38.151809 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0e591-c7a7-4956-9dcb-6f47eb82887d service swift] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:38.152147 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0e591-c7a7-4956-9dcb-6f47eb82887d service swift] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:38.152914 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0e591-c7a7-4956-9dcb-6f47eb82887d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:38.163898 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1371/2716] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:52:38 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:52:38.165428 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a171e0b-c522-4fb8-8a32-0528e847e784 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:38.169863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0e591-c7a7-4956-9dcb-6f47eb82887d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:38.169863 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0e591-c7a7-4956-9dcb-6f47eb82887d service swift] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:38.185662 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1346/2717] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Jul 25 12:52:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:52:38.268297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a171e0b-c522-4fb8-8a32-0528e847e784 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:38.275215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761f7c0-15f6-4e0d-b16c-3413ecf95a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:38.276061 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761f7c0-15f6-4e0d-b16c-3413ecf95a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:38.276477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761f7c0-15f6-4e0d-b16c-3413ecf95a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:38.277007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761f7c0-15f6-4e0d-b16c-3413ecf95a04 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:38.278011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761f7c0-15f6-4e0d-b16c-3413ecf95a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:38.280322 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761f7c0-15f6-4e0d-b16c-3413ecf95a04 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:38.314094 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1372/2718] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:52:38 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:52:38.315508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0e591-c7a7-4956-9dcb-6f47eb82887d service swift] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:38.349243 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0e591-c7a7-4956-9dcb-6f47eb82887d service swift] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:38.356475 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f059e-d3f0-4a2c-81d8-293e826a38ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:38.357151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f059e-d3f0-4a2c-81d8-293e826a38ca admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:38.357622 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f059e-d3f0-4a2c-81d8-293e826a38ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:38.357937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f059e-d3f0-4a2c-81d8-293e826a38ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:38.358697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f059e-d3f0-4a2c-81d8-293e826a38ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:38.360539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f059e-d3f0-4a2c-81d8-293e826a38ca admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:38.374648 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1347/2719] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:52:38.377968 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761f7c0-15f6-4e0d-b16c-3413ecf95a04 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:38.388951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761f7c0-15f6-4e0d-b16c-3413ecf95a04 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:38.402113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dffc7-907d-49f6-9652-c5d465c00048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:38.403039 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dffc7-907d-49f6-9652-c5d465c00048 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:38.403282 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dffc7-907d-49f6-9652-c5d465c00048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:38.403630 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dffc7-907d-49f6-9652-c5d465c00048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:38.404445 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4dffc7-907d-49f6-9652-c5d465c00048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:38.406740 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4dffc7-907d-49f6-9652-c5d465c00048 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:38.425447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1373/2720] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:52:38.429780 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f059e-d3f0-4a2c-81d8-293e826a38ca admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:38.445272 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f059e-d3f0-4a2c-81d8-293e826a38ca admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:38.455019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85040b1e-ff45-4076-8488-162e4b81dfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:38.455946 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85040b1e-ff45-4076-8488-162e4b81dfbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:38.456558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85040b1e-ff45-4076-8488-162e4b81dfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:38.456558 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85040b1e-ff45-4076-8488-162e4b81dfbe admin admin] PATH_INFO: `/v3/projects/b7a69b02cd124acdb14727ea0711c1f8/users/42b64d8fffbc4a0fa171099d5db580a0/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:38.480685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85040b1e-ff45-4076-8488-162e4b81dfbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a69b02cd124acdb14727ea0711c1f8, user_id=42b64d8fffbc4a0fa171099d5db580a0, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:38.483444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85040b1e-ff45-4076-8488-162e4b81dfbe admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:38.494886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:38.502029 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1348/2721] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:52:38 2022] PUT /identity/v3/projects/b7a69b02cd124acdb14727ea0711c1f8/users/42b64d8fffbc4a0fa171099d5db580a0/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:38.507259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f0451-89a8-4579-a2a6-6025dbe9e40b None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:38.507550 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f0451-89a8-4579-a2a6-6025dbe9e40b None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:38.507830 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f0451-89a8-4579-a2a6-6025dbe9e40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:38.534176 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-031f0451-89a8-4579-a2a6-6025dbe9e40b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:38.535495 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-031f0451-89a8-4579-a2a6-6025dbe9e40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:38.644846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1374/2722] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:38.663260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85040b1e-ff45-4076-8488-162e4b81dfbe admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:38.671145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-85040b1e-ff45-4076-8488-162e4b81dfbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:38.828939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85040b1e-ff45-4076-8488-162e4b81dfbe admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:38.835898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cab1f-240c-421e-b9b7-a9ecb83ca90b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:38.836552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cab1f-240c-421e-b9b7-a9ecb83ca90b service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:38.836887 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cab1f-240c-421e-b9b7-a9ecb83ca90b service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:38.837162 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cab1f-240c-421e-b9b7-a9ecb83ca90b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:38.849701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cab1f-240c-421e-b9b7-a9ecb83ca90b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:38.851970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cab1f-240c-421e-b9b7-a9ecb83ca90b service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:38.879749 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1349/2723] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:52:40.686893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f0451-89a8-4579-a2a6-6025dbe9e40b None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:40.688426 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-031f0451-89a8-4579-a2a6-6025dbe9e40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:40.708648 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f0451-89a8-4579-a2a6-6025dbe9e40b None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:40.714932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:40.716119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:40.716568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:40.717005 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] PATH_INFO: `/v3/users/b953dd2dc515426683501b4207839548` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:40.722891 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] RBAC: Authorizing `identity:delete_user(user_id=b953dd2dc515426683501b4207839548)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:40.725121 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:40.827658 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b953dd2dc515426683501b4207839548'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:40.841709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b953dd2dc515426683501b4207839548'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:40.842887 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b953dd2dc515426683501b4207839548'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:40.860749 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b953dd2dc515426683501b4207839548'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:40.864797 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b953dd2dc515426683501b4207839548'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:40.866998 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:40.923408 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1375/2724] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:52:40 2022] DELETE /identity/v3/users/b953dd2dc515426683501b4207839548 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:40.928321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fc238-e2db-4b28-a320-0b61360f3ca8 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:40.928823 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fc238-e2db-4b28-a320-0b61360f3ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:40.929108 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fc238-e2db-4b28-a320-0b61360f3ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:40.951104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-7a9fc238-e2db-4b28-a320-0b61360f3ca8 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:40.952779 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7a9fc238-e2db-4b28-a320-0b61360f3ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:41.061995 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1350/2725] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:41.109502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:41.111425 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:41.250997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04152568-aa97-4cf0-856c-9c003a2754be admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:41.257854 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33008c8-0ff9-441f-8833-1e969a3123c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:41.259962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33008c8-0ff9-441f-8833-1e969a3123c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:41.260690 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33008c8-0ff9-441f-8833-1e969a3123c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:41.261362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33008c8-0ff9-441f-8833-1e969a3123c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:41.293752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33008c8-0ff9-441f-8833-1e969a3123c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:41.296276 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33008c8-0ff9-441f-8833-1e969a3123c7 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:41.309932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1376/2726] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:52:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:52:41.759721 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fc238-e2db-4b28-a320-0b61360f3ca8 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:41.762028 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7a9fc238-e2db-4b28-a320-0b61360f3ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:41.782050 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fc238-e2db-4b28-a320-0b61360f3ca8 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:41.787330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269dced1-1d9d-454f-b733-4a39633ce1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:41.788219 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269dced1-1d9d-454f-b733-4a39633ce1c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:41.788504 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269dced1-1d9d-454f-b733-4a39633ce1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:41.788779 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269dced1-1d9d-454f-b733-4a39633ce1c2 admin admin] PATH_INFO: `/v3/projects/b678cb6fa6b340ae9cfc877a240ab213` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:41.790306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269dced1-1d9d-454f-b733-4a39633ce1c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b678cb6fa6b340ae9cfc877a240ab213)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:41.792305 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269dced1-1d9d-454f-b733-4a39633ce1c2 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:41.851850 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-269dced1-1d9d-454f-b733-4a39633ce1c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b678cb6fa6b340ae9cfc877a240ab213', 'request_id': 'req-269dced1-1d9d-454f-b733-4a39633ce1c2'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:41.859935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-269dced1-1d9d-454f-b733-4a39633ce1c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b678cb6fa6b340ae9cfc877a240ab213', 'request_id': 'req-269dced1-1d9d-454f-b733-4a39633ce1c2'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:41.865854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-269dced1-1d9d-454f-b733-4a39633ce1c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b678cb6fa6b340ae9cfc877a240ab213', 'request_id': 'req-269dced1-1d9d-454f-b733-4a39633ce1c2'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:41.882350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-269dced1-1d9d-454f-b733-4a39633ce1c2 admin admin] The token cache is being invalidate because project b678cb6fa6b340ae9cfc877a240ab213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:52:41.882350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-269dced1-1d9d-454f-b733-4a39633ce1c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:41.883060 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-269dced1-1d9d-454f-b733-4a39633ce1c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:52:41.884815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1351/2727] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:52:41 2022] DELETE /identity/v3/projects/b678cb6fa6b340ae9cfc877a240ab213 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:41.890950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28adba-a3a8-49be-8e31-be61f3492f0e None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:41.891404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28adba-a3a8-49be-8e31-be61f3492f0e None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:41.891404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28adba-a3a8-49be-8e31-be61f3492f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:41.914892 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9d28adba-a3a8-49be-8e31-be61f3492f0e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:41.915981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9d28adba-a3a8-49be-8e31-be61f3492f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:42.035833 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1377/2728] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:42.045405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b9bea-b166-4b54-b83b-d910f5ecc64e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:42.046114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b9bea-b166-4b54-b83b-d910f5ecc64e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:42.046340 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b9bea-b166-4b54-b83b-d910f5ecc64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:42.090380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-f64b9bea-b166-4b54-b83b-d910f5ecc64e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:42.093678 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f64b9bea-b166-4b54-b83b-d910f5ecc64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:42.106289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1352/2729] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:52:42.112409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28adba-a3a8-49be-8e31-be61f3492f0e None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:42.126019 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28adba-a3a8-49be-8e31-be61f3492f0e None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:42.131449 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1c200-a818-4d8d-a696-7360c065dd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:42.132624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1c200-a818-4d8d-a696-7360c065dd61 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:42.133044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1c200-a818-4d8d-a696-7360c065dd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:42.133555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1c200-a818-4d8d-a696-7360c065dd61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:42.134746 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1c200-a818-4d8d-a696-7360c065dd61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:52:42.135234 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1c200-a818-4d8d-a696-7360c065dd61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:42.137432 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1c200-a818-4d8d-a696-7360c065dd61 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:42.158909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1378/2730] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:52:42.161503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b9bea-b166-4b54-b83b-d910f5ecc64e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:42.192575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b9bea-b166-4b54-b83b-d910f5ecc64e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:42.218033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7175bdc-ee92-4baa-bee3-319f09322126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:42.218857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7175bdc-ee92-4baa-bee3-319f09322126 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:42.219148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7175bdc-ee92-4baa-bee3-319f09322126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:42.219425 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7175bdc-ee92-4baa-bee3-319f09322126 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:42.220177 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7175bdc-ee92-4baa-bee3-319f09322126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:42.222274 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7175bdc-ee92-4baa-bee3-319f09322126 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:42.267395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:42.286886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1353/2731] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:52:42 2022] POST /identity/v3/projects => generated 391 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:52:42.291139 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1c200-a818-4d8d-a696-7360c065dd61 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:42.304365 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1c200-a818-4d8d-a696-7360c065dd61 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:42.403935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade4284-49e6-4212-bb87-411483461ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:42.404962 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade4284-49e6-4212-bb87-411483461ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:42.405259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade4284-49e6-4212-bb87-411483461ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:42.405529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade4284-49e6-4212-bb87-411483461ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:42.406340 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade4284-49e6-4212-bb87-411483461ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:42.408339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade4284-49e6-4212-bb87-411483461ca4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:42.439312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:42.450991 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1379/2732] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:52:42 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:52:42.452238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7175bdc-ee92-4baa-bee3-319f09322126 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:42.465180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7175bdc-ee92-4baa-bee3-319f09322126 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:42.471684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af772f5-5e8f-46ff-99a9-757bbeda5629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:42.473321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af772f5-5e8f-46ff-99a9-757bbeda5629 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:42.473635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af772f5-5e8f-46ff-99a9-757bbeda5629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:42.473941 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af772f5-5e8f-46ff-99a9-757bbeda5629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:42.475978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af772f5-5e8f-46ff-99a9-757bbeda5629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:42.480552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af772f5-5e8f-46ff-99a9-757bbeda5629 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:42.502745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1354/2733] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:52:42.503978 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade4284-49e6-4212-bb87-411483461ca4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:42.517473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade4284-49e6-4212-bb87-411483461ca4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:42.525315 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99b99e-3060-44e8-83d3-97ff67906557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:42.527011 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99b99e-3060-44e8-83d3-97ff67906557 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:42.527367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99b99e-3060-44e8-83d3-97ff67906557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:42.527653 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99b99e-3060-44e8-83d3-97ff67906557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:42.542880 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99b99e-3060-44e8-83d3-97ff67906557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:42.545118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99b99e-3060-44e8-83d3-97ff67906557 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:42.560770 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1380/2734] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:52:42.562228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af772f5-5e8f-46ff-99a9-757bbeda5629 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:42.573405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af772f5-5e8f-46ff-99a9-757bbeda5629 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:42.578918 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514f60-93b2-4e12-8f5e-a8d5adb31a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:42.579827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514f60-93b2-4e12-8f5e-a8d5adb31a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:42.580114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514f60-93b2-4e12-8f5e-a8d5adb31a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:42.580399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514f60-93b2-4e12-8f5e-a8d5adb31a5a admin admin] PATH_INFO: `/v3/projects/7cefde75862249d6b654ec922771c163/users/f50cf2f6516b4ed79e65600f844ae35d/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:42.599529 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e514f60-93b2-4e12-8f5e-a8d5adb31a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cefde75862249d6b654ec922771c163, user_id=f50cf2f6516b4ed79e65600f844ae35d, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:42.602418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e514f60-93b2-4e12-8f5e-a8d5adb31a5a admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:42.621566 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1355/2735] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:52:42 2022] PUT /identity/v3/projects/7cefde75862249d6b654ec922771c163/users/f50cf2f6516b4ed79e65600f844ae35d/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:42.632036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17852250-9c2e-4224-bddf-126033a9d3b3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:42.632383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17852250-9c2e-4224-bddf-126033a9d3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:42.632607 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17852250-9c2e-4224-bddf-126033a9d3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:42.684128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-17852250-9c2e-4224-bddf-126033a9d3b3 None None] MFA Rules not processed for user `f50cf2f6516b4ed79e65600f844ae35d`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:42.686977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-17852250-9c2e-4224-bddf-126033a9d3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:42.768010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1381/2736] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:42.779544 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514f60-93b2-4e12-8f5e-a8d5adb31a5a admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:42.781509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-9e514f60-93b2-4e12-8f5e-a8d5adb31a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:42.851583 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514f60-93b2-4e12-8f5e-a8d5adb31a5a admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:42.857100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b27f5-fd18-4082-a5dd-336a61ac8fc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:42.857787 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b27f5-fd18-4082-a5dd-336a61ac8fc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:42.858139 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b27f5-fd18-4082-a5dd-336a61ac8fc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:42.858560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b27f5-fd18-4082-a5dd-336a61ac8fc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:42.869499 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b27f5-fd18-4082-a5dd-336a61ac8fc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:42.871625 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b27f5-fd18-4082-a5dd-336a61ac8fc8 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:42.894832 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1356/2737] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:52:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:52:43.503714 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c573261-c00a-4d65-88de-742fcd14d82c None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:43.504283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c573261-c00a-4d65-88de-742fcd14d82c None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:43.504283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c573261-c00a-4d65-88de-742fcd14d82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:43.534001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-2c573261-c00a-4d65-88de-742fcd14d82c None None] MFA Rules not processed for user `9a4b1f0404e74e0e9e53e25ad92dcfed`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:43.535289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2c573261-c00a-4d65-88de-742fcd14d82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:43.620579 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1382/2738] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:44.395117 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153f12e-3d52-4aad-a0ba-e1d940941297 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:44.395649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153f12e-3d52-4aad-a0ba-e1d940941297 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:44.395649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153f12e-3d52-4aad-a0ba-e1d940941297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:44.461600 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-8153f12e-3d52-4aad-a0ba-e1d940941297 None None] MFA Rules not processed for user `42b64d8fffbc4a0fa171099d5db580a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:44.462875 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8153f12e-3d52-4aad-a0ba-e1d940941297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:44.581038 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1357/2739] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:44.591016 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c573261-c00a-4d65-88de-742fcd14d82c None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:44.593150 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-2c573261-c00a-4d65-88de-742fcd14d82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:44.705606 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c573261-c00a-4d65-88de-742fcd14d82c None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:44.711964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975105b-301c-4a9b-83ac-ccbe9241138c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:44.713299 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975105b-301c-4a9b-83ac-ccbe9241138c admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:44.714380 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975105b-301c-4a9b-83ac-ccbe9241138c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:44.714933 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975105b-301c-4a9b-83ac-ccbe9241138c admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:44.715761 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975105b-301c-4a9b-83ac-ccbe9241138c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:44.718056 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975105b-301c-4a9b-83ac-ccbe9241138c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:44.776695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153f12e-3d52-4aad-a0ba-e1d940941297 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:44.777715 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1383/2740] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:52:44 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:52:44.788579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153f12e-3d52-4aad-a0ba-e1d940941297 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:44.871014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:44.898808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8ed7f-8570-4107-927c-ae3d29bb8361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:44.899632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8ed7f-8570-4107-927c-ae3d29bb8361 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:44.900021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8ed7f-8570-4107-927c-ae3d29bb8361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:44.901848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8ed7f-8570-4107-927c-ae3d29bb8361 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:44.902785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8ed7f-8570-4107-927c-ae3d29bb8361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:44.905034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8ed7f-8570-4107-927c-ae3d29bb8361 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:44.929846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1358/2741] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:52:44 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:52:44.934003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975105b-301c-4a9b-83ac-ccbe9241138c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:44.953897 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975105b-301c-4a9b-83ac-ccbe9241138c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:44.959117 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87909f4-83d9-471b-ae16-16ebcae01c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:44.959979 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87909f4-83d9-471b-ae16-16ebcae01c6c admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:44.960307 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87909f4-83d9-471b-ae16-16ebcae01c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:44.960655 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87909f4-83d9-471b-ae16-16ebcae01c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:44.961502 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87909f4-83d9-471b-ae16-16ebcae01c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:44.963346 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87909f4-83d9-471b-ae16-16ebcae01c6c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:44.966928 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:52:44.983923 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1384/2742] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:52:44.988552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8ed7f-8570-4107-927c-ae3d29bb8361 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:44.999969 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8ed7f-8570-4107-927c-ae3d29bb8361 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:45.005029 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba0cf3-4567-4278-b585-1d877b0e06cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:45.005988 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba0cf3-4567-4278-b585-1d877b0e06cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:45.006790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba0cf3-4567-4278-b585-1d877b0e06cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:45.006790 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba0cf3-4567-4278-b585-1d877b0e06cb admin admin] PATH_INFO: `/v3/projects/148c9c2c71984436abc9f275e4f890c3/users/6059757c96c2499ba82ef0d39a1a6355/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:45.024364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba0cf3-4567-4278-b585-1d877b0e06cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=148c9c2c71984436abc9f275e4f890c3, user_id=6059757c96c2499ba82ef0d39a1a6355, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:45.027667 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba0cf3-4567-4278-b585-1d877b0e06cb admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:45.048594 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1359/2743] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:52:44 2022] PUT /identity/v3/projects/148c9c2c71984436abc9f275e4f890c3/users/6059757c96c2499ba82ef0d39a1a6355/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:52:45.052857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87909f4-83d9-471b-ae16-16ebcae01c6c admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:45.064374 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87909f4-83d9-471b-ae16-16ebcae01c6c admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:45.137382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f56d7-c9d8-4a3d-aa4d-9d4b4d3ffb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:45.138030 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f56d7-c9d8-4a3d-aa4d-9d4b4d3ffb0e admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:45.138317 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f56d7-c9d8-4a3d-aa4d-9d4b4d3ffb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:45.138595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f56d7-c9d8-4a3d-aa4d-9d4b4d3ffb0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:45.139337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f56d7-c9d8-4a3d-aa4d-9d4b4d3ffb0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:45.140959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f56d7-c9d8-4a3d-aa4d-9d4b4d3ffb0e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:45.154472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1385/2744] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:52:45.158701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba0cf3-4567-4278-b585-1d877b0e06cb admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:45.171532 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba0cf3-4567-4278-b585-1d877b0e06cb admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:45.179512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac714f3-9e43-43b6-b1d7-58a7992dadbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:45.185493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac714f3-9e43-43b6-b1d7-58a7992dadbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:45.185493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac714f3-9e43-43b6-b1d7-58a7992dadbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:45.185493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac714f3-9e43-43b6-b1d7-58a7992dadbe admin admin] PATH_INFO: `/v3/projects/148c9c2c71984436abc9f275e4f890c3/users/6059757c96c2499ba82ef0d39a1a6355/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:45.185493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac714f3-9e43-43b6-b1d7-58a7992dadbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=148c9c2c71984436abc9f275e4f890c3, user_id=6059757c96c2499ba82ef0d39a1a6355, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:45.189034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac714f3-9e43-43b6-b1d7-58a7992dadbe admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:45.203073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1360/2745] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:52:45 2022] PUT /identity/v3/projects/148c9c2c71984436abc9f275e4f890c3/users/6059757c96c2499ba82ef0d39a1a6355/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:52:50.409105 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c5f5c-7099-4b17-b08d-c321d1cabafa None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:50.410741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c5f5c-7099-4b17-b08d-c321d1cabafa None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:50.410741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c5f5c-7099-4b17-b08d-c321d1cabafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:50.445866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-c35c5f5c-7099-4b17-b08d-c321d1cabafa None None] MFA Rules not processed for user `6059757c96c2499ba82ef0d39a1a6355`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:50.447381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c35c5f5c-7099-4b17-b08d-c321d1cabafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:50.543040 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1386/2746] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:52:50.551388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac714f3-9e43-43b6-b1d7-58a7992dadbe admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:52:50.554283 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-bac714f3-9e43-43b6-b1d7-58a7992dadbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:50.690331 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac714f3-9e43-43b6-b1d7-58a7992dadbe admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:52:50.697083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc120e3-7fda-4505-8f0c-2460930a991b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:52:50.697760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc120e3-7fda-4505-8f0c-2460930a991b service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:50.698115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc120e3-7fda-4505-8f0c-2460930a991b service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:50.698390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc120e3-7fda-4505-8f0c-2460930a991b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:50.777320 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc120e3-7fda-4505-8f0c-2460930a991b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:52:50.779872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc120e3-7fda-4505-8f0c-2460930a991b service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:52:50.795231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1361/2747] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:52:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:52:52.308357 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa6cce-b70f-476c-9489-859f844ec6fb None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:52:52.308869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa6cce-b70f-476c-9489-859f844ec6fb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:52:52.308967 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa6cce-b70f-476c-9489-859f844ec6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:52:52.328229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-b5fa6cce-b70f-476c-9489-859f844ec6fb None None] MFA Rules not processed for user `42b64d8fffbc4a0fa171099d5db580a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:52:52.329457 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-b5fa6cce-b70f-476c-9489-859f844ec6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:52:52.336571 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1387/2748] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:52:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:53:00.788997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:53:02.323323 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=67 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:53:09.104806 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc120e3-7fda-4505-8f0c-2460930a991b service nova] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:09.166604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc120e3-7fda-4505-8f0c-2460930a991b service nova] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:09.171642 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:09.172520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:09.172808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:09.173070 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] PATH_INFO: `/v3/users/f50cf2f6516b4ed79e65600f844ae35d` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:09.174283 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f50cf2f6516b4ed79e65600f844ae35d)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:09.176215 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:09.224962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f50cf2f6516b4ed79e65600f844ae35d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:09.225613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f50cf2f6516b4ed79e65600f844ae35d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:09.226225 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f50cf2f6516b4ed79e65600f844ae35d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:09.234195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f50cf2f6516b4ed79e65600f844ae35d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:09.242896 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f50cf2f6516b4ed79e65600f844ae35d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:09.261237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1362/2749] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:53:09 2022] DELETE /identity/v3/users/f50cf2f6516b4ed79e65600f844ae35d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:53:09.265950 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed36b5-2679-4528-81a9-2e3bdb0964db None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:09.266480 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed36b5-2679-4528-81a9-2e3bdb0964db None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:09.266603 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed36b5-2679-4528-81a9-2e3bdb0964db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:09.286242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-21ed36b5-2679-4528-81a9-2e3bdb0964db None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:53:09.288131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-21ed36b5-2679-4528-81a9-2e3bdb0964db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:09.353965 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1388/2750] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:53:09.364522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:09.365958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:09.424551 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e007ee3-e163-4b1a-9faf-57dfd5505c0b admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:09.428186 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570cbaf-1de0-4958-a7df-cd3209388ac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:09.428829 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570cbaf-1de0-4958-a7df-cd3209388ac0 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:09.429106 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570cbaf-1de0-4958-a7df-cd3209388ac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:09.429380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570cbaf-1de0-4958-a7df-cd3209388ac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:09.438237 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570cbaf-1de0-4958-a7df-cd3209388ac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:09.439648 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570cbaf-1de0-4958-a7df-cd3209388ac0 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:09.448821 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1363/2751] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:53:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:53:09.730465 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed36b5-2679-4528-81a9-2e3bdb0964db None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:09.731932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-21ed36b5-2679-4528-81a9-2e3bdb0964db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:09.744840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed36b5-2679-4528-81a9-2e3bdb0964db None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:09.750215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:09.751257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:09.751543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:09.751818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe admin admin] PATH_INFO: `/v3/projects/7cefde75862249d6b654ec922771c163` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:09.753250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cefde75862249d6b654ec922771c163)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:09.756404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:09.803357 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cefde75862249d6b654ec922771c163', 'request_id': 'req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:09.810473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cefde75862249d6b654ec922771c163', 'request_id': 'req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:09.816825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cefde75862249d6b654ec922771c163', 'request_id': 'req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:09.827515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe admin admin] The token cache is being invalidate because project 7cefde75862249d6b654ec922771c163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:53:09.827805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:09.828426 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d3f91f61-ad4c-4ff3-9cba-5b1051b660fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:09.829977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1389/2752] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:53:09 2022] DELETE /identity/v3/projects/7cefde75862249d6b654ec922771c163 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:53:09.834857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc541df-00b0-4fad-b887-d40daf28dc6e None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:09.835144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc541df-00b0-4fad-b887-d40daf28dc6e None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:09.835449 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc541df-00b0-4fad-b887-d40daf28dc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:09.857584 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-dcc541df-00b0-4fad-b887-d40daf28dc6e None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:53:09.858818 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-dcc541df-00b0-4fad-b887-d40daf28dc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:09.876900 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:53:09.922444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1364/2753] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:53:09.931295 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9648a-f339-4049-a7d6-4e69fb959c04 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:09.931584 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9648a-f339-4049-a7d6-4e69fb959c04 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:09.931866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9648a-f339-4049-a7d6-4e69fb959c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:09.951383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ecc9648a-f339-4049-a7d6-4e69fb959c04 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:53:09.953174 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ecc9648a-f339-4049-a7d6-4e69fb959c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:09.961272 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1390/2754] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:53:09.965715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc541df-00b0-4fad-b887-d40daf28dc6e None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:09.976401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc541df-00b0-4fad-b887-d40daf28dc6e None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:09.982400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98ca1f-4782-4a1f-9eea-879ad215c8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:09.983081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98ca1f-4782-4a1f-9eea-879ad215c8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:09.983365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98ca1f-4782-4a1f-9eea-879ad215c8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:09.983644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98ca1f-4782-4a1f-9eea-879ad215c8c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:09.984388 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98ca1f-4782-4a1f-9eea-879ad215c8c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:53:09.984990 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98ca1f-4782-4a1f-9eea-879ad215c8c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:09.986793 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98ca1f-4782-4a1f-9eea-879ad215c8c4 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:10.000807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1365/2755] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:53:10.005005 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9648a-f339-4049-a7d6-4e69fb959c04 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:10.015104 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9648a-f339-4049-a7d6-4e69fb959c04 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:10.020072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9073ac-e382-4a86-9465-fddc6f5c94e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:10.020907 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9073ac-e382-4a86-9465-fddc6f5c94e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:10.021193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9073ac-e382-4a86-9465-fddc6f5c94e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:10.021467 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9073ac-e382-4a86-9465-fddc6f5c94e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:10.022210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9073ac-e382-4a86-9465-fddc6f5c94e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:10.024131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9073ac-e382-4a86-9465-fddc6f5c94e8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:10.057151 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1391/2756] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:53:10 2022] POST /identity/v3/projects => generated 397 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:53:10.063511 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98ca1f-4782-4a1f-9eea-879ad215c8c4 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:10.073760 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98ca1f-4782-4a1f-9eea-879ad215c8c4 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:10.141580 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb6b66-e79b-40ec-b679-79aef9880e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:10.142607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb6b66-e79b-40ec-b679-79aef9880e63 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:10.142607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb6b66-e79b-40ec-b679-79aef9880e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:10.142923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb6b66-e79b-40ec-b679-79aef9880e63 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:10.143796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb6b66-e79b-40ec-b679-79aef9880e63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:10.145869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb6b66-e79b-40ec-b679-79aef9880e63 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:10.168347 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1366/2757] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:53:10 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:53:10.172583 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9073ac-e382-4a86-9465-fddc6f5c94e8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:10.192574 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9073ac-e382-4a86-9465-fddc6f5c94e8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:10.197287 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370c162-1516-4cb3-adc3-046e2f4efd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:10.197986 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370c162-1516-4cb3-adc3-046e2f4efd61 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:10.198303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370c162-1516-4cb3-adc3-046e2f4efd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:10.198581 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370c162-1516-4cb3-adc3-046e2f4efd61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:10.199316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370c162-1516-4cb3-adc3-046e2f4efd61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:10.200927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370c162-1516-4cb3-adc3-046e2f4efd61 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:10.211989 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1392/2758] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:53:10.216075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb6b66-e79b-40ec-b679-79aef9880e63 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:10.226508 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb6b66-e79b-40ec-b679-79aef9880e63 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:10.234223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2745376-b34f-4cab-87e5-4b96e9772625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:10.235170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2745376-b34f-4cab-87e5-4b96e9772625 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:10.235627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2745376-b34f-4cab-87e5-4b96e9772625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:10.236057 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2745376-b34f-4cab-87e5-4b96e9772625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:10.236828 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2745376-b34f-4cab-87e5-4b96e9772625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:10.238690 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2745376-b34f-4cab-87e5-4b96e9772625 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:10.249624 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1367/2759] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:53:10.253901 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370c162-1516-4cb3-adc3-046e2f4efd61 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:10.265771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370c162-1516-4cb3-adc3-046e2f4efd61 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:10.270619 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13827653-0fcf-4a39-965f-b619f359cedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:10.271871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827653-0fcf-4a39-965f-b619f359cedf admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:10.272243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827653-0fcf-4a39-965f-b619f359cedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:10.272635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827653-0fcf-4a39-965f-b619f359cedf admin admin] PATH_INFO: `/v3/projects/717c8e67d5864b5f959d1b995e4b928c/users/61f82b233d6148d3ab773b7d0626679c/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:10.290063 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13827653-0fcf-4a39-965f-b619f359cedf admin admin] RBAC: Authorizing `identity:create_grant(project_id=717c8e67d5864b5f959d1b995e4b928c, user_id=61f82b233d6148d3ab773b7d0626679c, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:10.292833 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13827653-0fcf-4a39-965f-b619f359cedf admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:10.307138 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1393/2760] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:53:10 2022] PUT /identity/v3/projects/717c8e67d5864b5f959d1b995e4b928c/users/61f82b233d6148d3ab773b7d0626679c/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:53:10.318322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e878934-f6f6-4e11-af55-974f924bb0e0 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:10.318611 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e878934-f6f6-4e11-af55-974f924bb0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:10.318889 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e878934-f6f6-4e11-af55-974f924bb0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:10.350739 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-8e878934-f6f6-4e11-af55-974f924bb0e0 None None] MFA Rules not processed for user `61f82b233d6148d3ab773b7d0626679c`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:53:10.352196 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8e878934-f6f6-4e11-af55-974f924bb0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:10.425720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1368/2761] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:53:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:53:10.437874 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13827653-0fcf-4a39-965f-b619f359cedf admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:10.439632 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-13827653-0fcf-4a39-965f-b619f359cedf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:10.461231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:53:10.507540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13827653-0fcf-4a39-965f-b619f359cedf admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:10.511271 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f44360-5afa-4d44-b946-1902b911592c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:10.511997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f44360-5afa-4d44-b946-1902b911592c service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:10.512318 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f44360-5afa-4d44-b946-1902b911592c service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:10.512698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f44360-5afa-4d44-b946-1902b911592c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:10.522638 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f44360-5afa-4d44-b946-1902b911592c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:10.524119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f44360-5afa-4d44-b946-1902b911592c service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:10.533423 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1394/2762] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:53:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:53:12.246626 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e878934-f6f6-4e11-af55-974f924bb0e0 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:12.248018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8e878934-f6f6-4e11-af55-974f924bb0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:12.268775 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:53:12.326519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e878934-f6f6-4e11-af55-974f924bb0e0 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:12.332703 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e600855-9ca9-4fa0-a6e7-497cfd459aa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:12.333356 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e600855-9ca9-4fa0-a6e7-497cfd459aa7 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:12.333634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e600855-9ca9-4fa0-a6e7-497cfd459aa7 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:12.333905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e600855-9ca9-4fa0-a6e7-497cfd459aa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:12.335685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2e600855-9ca9-4fa0-a6e7-497cfd459aa7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:12.348083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e600855-9ca9-4fa0-a6e7-497cfd459aa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:12.349657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e600855-9ca9-4fa0-a6e7-497cfd459aa7 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:12.360923 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1369/2763] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:53:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:53:20.529360 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=17 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:53:20.741098 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f44360-5afa-4d44-b946-1902b911592c service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:20.807808 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f44360-5afa-4d44-b946-1902b911592c service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:20.812492 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:20.813406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:20.813687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:20.813994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] PATH_INFO: `/v3/users/61f82b233d6148d3ab773b7d0626679c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:20.815296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f82b233d6148d3ab773b7d0626679c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:20.817242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:20.866006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f82b233d6148d3ab773b7d0626679c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:20.873614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f82b233d6148d3ab773b7d0626679c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:20.874249 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f82b233d6148d3ab773b7d0626679c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:20.881311 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f82b233d6148d3ab773b7d0626679c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:20.882027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f82b233d6148d3ab773b7d0626679c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:20.899835 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1395/2764] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:53:20 2022] DELETE /identity/v3/users/61f82b233d6148d3ab773b7d0626679c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:53:20.904520 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a58cb1-9ebe-4427-abd7-624c5ca89d76 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:20.905570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a58cb1-9ebe-4427-abd7-624c5ca89d76 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:20.905677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a58cb1-9ebe-4427-abd7-624c5ca89d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:20.930178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-81a58cb1-9ebe-4427-abd7-624c5ca89d76 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:53:20.932151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-81a58cb1-9ebe-4427-abd7-624c5ca89d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:20.993295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1370/2765] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:53:21.003297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:21.004830 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:21.062822 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d29fe-8b43-43e4-993a-2c484fed8ccf admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:21.066418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503f76f-75ee-4949-a5a0-f460e6aa3bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:21.067073 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503f76f-75ee-4949-a5a0-f460e6aa3bfc service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:21.067357 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503f76f-75ee-4949-a5a0-f460e6aa3bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:21.067635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503f76f-75ee-4949-a5a0-f460e6aa3bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:21.077850 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503f76f-75ee-4949-a5a0-f460e6aa3bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:21.079332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503f76f-75ee-4949-a5a0-f460e6aa3bfc service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:21.089152 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1396/2766] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:53:21.391798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a58cb1-9ebe-4427-abd7-624c5ca89d76 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:21.393239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-81a58cb1-9ebe-4427-abd7-624c5ca89d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:21.405323 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a58cb1-9ebe-4427-abd7-624c5ca89d76 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:21.411085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:21.412515 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:21.412922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:21.413205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] PATH_INFO: `/v3/projects/717c8e67d5864b5f959d1b995e4b928c` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:21.414510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=717c8e67d5864b5f959d1b995e4b928c)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:21.417101 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:21.471825 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '717c8e67d5864b5f959d1b995e4b928c', 'request_id': 'req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:21.478151 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '717c8e67d5864b5f959d1b995e4b928c', 'request_id': 'req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:21.483668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '717c8e67d5864b5f959d1b995e4b928c', 'request_id': 'req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:21.497384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] The token cache is being invalidate because project 717c8e67d5864b5f959d1b995e4b928c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:53:21.497801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:21.499115 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:53:21.500759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1371/2767] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:53:21 2022] DELETE /identity/v3/projects/717c8e67d5864b5f959d1b995e4b928c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:53:23.720803 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503f76f-75ee-4949-a5a0-f460e6aa3bfc service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:23.729599 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503f76f-75ee-4949-a5a0-f460e6aa3bfc service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:23.750245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:53:23.784303 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82e214-3b24-4352-9321-6e644e923a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:23.784959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82e214-3b24-4352-9321-6e644e923a3d service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:23.785247 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82e214-3b24-4352-9321-6e644e923a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:23.785527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82e214-3b24-4352-9321-6e644e923a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:23.787351 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1a82e214-3b24-4352-9321-6e644e923a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:23.798057 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82e214-3b24-4352-9321-6e644e923a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:23.799525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82e214-3b24-4352-9321-6e644e923a3d service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:23.808125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1397/2768] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:53:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:53:31.493808 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:53:33.804213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=16 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:53:41.035976 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:41.037432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:41.056238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea59d6-eebb-48bc-83e9-a5bfc09f7e52 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:41.059833 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14a35b-2ac3-4282-8685-b1e4978d00e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:41.060481 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14a35b-2ac3-4282-8685-b1e4978d00e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:41.060765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14a35b-2ac3-4282-8685-b1e4978d00e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:41.061041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14a35b-2ac3-4282-8685-b1e4978d00e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:41.062856 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7f14a35b-2ac3-4282-8685-b1e4978d00e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:41.130152 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14a35b-2ac3-4282-8685-b1e4978d00e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:41.131596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14a35b-2ac3-4282-8685-b1e4978d00e8 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:41.141591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1372/2769] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:53:41 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:53:42.418678 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82e214-3b24-4352-9321-6e644e923a3d service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:53:42.420182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1a82e214-3b24-4352-9321-6e644e923a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:42.494021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82e214-3b24-4352-9321-6e644e923a3d service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:53:42.499227 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685c79f-8c8b-49bf-a578-b66ba583f3b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:53:42.499914 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685c79f-8c8b-49bf-a578-b66ba583f3b8 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:53:42.500201 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685c79f-8c8b-49bf-a578-b66ba583f3b8 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:53:42.500478 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685c79f-8c8b-49bf-a578-b66ba583f3b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:53:42.502535 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f685c79f-8c8b-49bf-a578-b66ba583f3b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:53:42.516194 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685c79f-8c8b-49bf-a578-b66ba583f3b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:53:42.518435 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685c79f-8c8b-49bf-a578-b66ba583f3b8 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:53:42.529819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1398/2770] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:53:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:53:51.137127 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=30 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:53:52.525059 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=32 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:54:37.958659 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14a35b-2ac3-4282-8685-b1e4978d00e8 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:54:37.960159 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-7f14a35b-2ac3-4282-8685-b1e4978d00e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:54:38.021601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14a35b-2ac3-4282-8685-b1e4978d00e8 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:54:38.025278 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9969a-20b9-40f2-b7ec-b8aff366c813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:54:38.025939 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9969a-20b9-40f2-b7ec-b8aff366c813 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:54:38.026229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9969a-20b9-40f2-b7ec-b8aff366c813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:54:38.026501 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9969a-20b9-40f2-b7ec-b8aff366c813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:54:38.028326 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-65c9969a-20b9-40f2-b7ec-b8aff366c813 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:54:38.100042 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9969a-20b9-40f2-b7ec-b8aff366c813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:54:38.101470 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9969a-20b9-40f2-b7ec-b8aff366c813 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:54:38.113929 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1373/2771] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:54:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:54:46.176764 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685c79f-8c8b-49bf-a578-b66ba583f3b8 service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:54:46.186556 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685c79f-8c8b-49bf-a578-b66ba583f3b8 service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:54:46.190731 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a52823-6017-4b61-85cd-f5cda1aab0c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:54:46.191468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a52823-6017-4b61-85cd-f5cda1aab0c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:54:46.191759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a52823-6017-4b61-85cd-f5cda1aab0c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:54:46.192042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a52823-6017-4b61-85cd-f5cda1aab0c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:54:46.193875 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e6a52823-6017-4b61-85cd-f5cda1aab0c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:54:46.209448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a52823-6017-4b61-85cd-f5cda1aab0c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:54:46.210952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a52823-6017-4b61-85cd-f5cda1aab0c9 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:54:46.223240 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1399/2772] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:54:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:54:48.108668 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=54 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:54:56.217386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:54:58.175552 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9969a-20b9-40f2-b7ec-b8aff366c813 service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:54:58.177008 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-65c9969a-20b9-40f2-b7ec-b8aff366c813 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:54:58.188724 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9969a-20b9-40f2-b7ec-b8aff366c813 service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:54:58.192474 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260caad3-d0a7-4f5a-af84-b82c34d94690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:54:58.193129 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260caad3-d0a7-4f5a-af84-b82c34d94690 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:54:58.193412 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260caad3-d0a7-4f5a-af84-b82c34d94690 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:54:58.193692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260caad3-d0a7-4f5a-af84-b82c34d94690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:54:58.195522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-260caad3-d0a7-4f5a-af84-b82c34d94690 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:54:58.259228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260caad3-d0a7-4f5a-af84-b82c34d94690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:54:58.260664 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260caad3-d0a7-4f5a-af84-b82c34d94690 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:54:58.271222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1374/2773] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:54:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:55:01.520869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a52823-6017-4b61-85cd-f5cda1aab0c9 service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:01.524341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e6a52823-6017-4b61-85cd-f5cda1aab0c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:01.542242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a52823-6017-4b61-85cd-f5cda1aab0c9 service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:01.547592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dda8c7-7714-4f8d-8f50-9c3003f1931f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:01.548854 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dda8c7-7714-4f8d-8f50-9c3003f1931f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:01.549145 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dda8c7-7714-4f8d-8f50-9c3003f1931f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:01.549531 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dda8c7-7714-4f8d-8f50-9c3003f1931f admin admin] PATH_INFO: `/v3/users/9a4b1f0404e74e0e9e53e25ad92dcfed` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:01.551010 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dda8c7-7714-4f8d-8f50-9c3003f1931f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a4b1f0404e74e0e9e53e25ad92dcfed)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:01.552975 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dda8c7-7714-4f8d-8f50-9c3003f1931f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:01.616623 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-84dda8c7-7714-4f8d-8f50-9c3003f1931f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a4b1f0404e74e0e9e53e25ad92dcfed'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:01.625540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-84dda8c7-7714-4f8d-8f50-9c3003f1931f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a4b1f0404e74e0e9e53e25ad92dcfed'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:01.626237 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-84dda8c7-7714-4f8d-8f50-9c3003f1931f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a4b1f0404e74e0e9e53e25ad92dcfed'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:01.638418 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-84dda8c7-7714-4f8d-8f50-9c3003f1931f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a4b1f0404e74e0e9e53e25ad92dcfed'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:01.639352 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-84dda8c7-7714-4f8d-8f50-9c3003f1931f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a4b1f0404e74e0e9e53e25ad92dcfed'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:01.661475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1400/2774] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:55:01 2022] DELETE /identity/v3/users/9a4b1f0404e74e0e9e53e25ad92dcfed => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:01.868205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260caad3-d0a7-4f5a-af84-b82c34d94690 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:01.879349 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260caad3-d0a7-4f5a-af84-b82c34d94690 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:01.949900 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b982c71-4cd6-406b-be59-06e94f208616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:01.950804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b982c71-4cd6-406b-be59-06e94f208616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:01.951088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b982c71-4cd6-406b-be59-06e94f208616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:01.951364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b982c71-4cd6-406b-be59-06e94f208616 admin admin] PATH_INFO: `/v3/projects/47ad60e5634942149abd3837dc89a019` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:01.952607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b982c71-4cd6-406b-be59-06e94f208616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ad60e5634942149abd3837dc89a019)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:01.955191 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b982c71-4cd6-406b-be59-06e94f208616 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:02.009405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2b982c71-4cd6-406b-be59-06e94f208616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ad60e5634942149abd3837dc89a019', 'request_id': 'req-2b982c71-4cd6-406b-be59-06e94f208616'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:02.016259 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2b982c71-4cd6-406b-be59-06e94f208616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ad60e5634942149abd3837dc89a019', 'request_id': 'req-2b982c71-4cd6-406b-be59-06e94f208616'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:02.021890 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2b982c71-4cd6-406b-be59-06e94f208616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ad60e5634942149abd3837dc89a019', 'request_id': 'req-2b982c71-4cd6-406b-be59-06e94f208616'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:02.035837 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2b982c71-4cd6-406b-be59-06e94f208616 admin admin] The token cache is being invalidate because project 47ad60e5634942149abd3837dc89a019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:55:02.036109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2b982c71-4cd6-406b-be59-06e94f208616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:02.036708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2b982c71-4cd6-406b-be59-06e94f208616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:02.038321 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1375/2775] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:55:01 2022] DELETE /identity/v3/projects/47ad60e5634942149abd3837dc89a019 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:02.043340 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f46635-d18b-41fc-b83f-5a8a16706212 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.043628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f46635-d18b-41fc-b83f-5a8a16706212 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.043908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f46635-d18b-41fc-b83f-5a8a16706212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.064791 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-52f46635-d18b-41fc-b83f-5a8a16706212 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:02.066044 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-52f46635-d18b-41fc-b83f-5a8a16706212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:02.131489 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1401/2776] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:55:02.139473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c768b-c08b-4fbc-b7f2-c65b4b22333b None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.139756 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c768b-c08b-4fbc-b7f2-c65b4b22333b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.140055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c768b-c08b-4fbc-b7f2-c65b4b22333b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.160396 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-fe3c768b-c08b-4fbc-b7f2-c65b4b22333b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:02.161644 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fe3c768b-c08b-4fbc-b7f2-c65b4b22333b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:02.170970 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1376/2777] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:02.176061 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f46635-d18b-41fc-b83f-5a8a16706212 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:02.189529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f46635-d18b-41fc-b83f-5a8a16706212 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:02.191793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1534475-f386-4ad3-abc9-9c887a799ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:02.192469 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534475-f386-4ad3-abc9-9c887a799ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.192756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534475-f386-4ad3-abc9-9c887a799ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.193036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534475-f386-4ad3-abc9-9c887a799ac1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.193778 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1534475-f386-4ad3-abc9-9c887a799ac1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:55:02.194119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1534475-f386-4ad3-abc9-9c887a799ac1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:02.195771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1534475-f386-4ad3-abc9-9c887a799ac1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:02.209505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1402/2778] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:55:02.214318 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c768b-c08b-4fbc-b7f2-c65b4b22333b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:02.223975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c768b-c08b-4fbc-b7f2-c65b4b22333b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:02.228601 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08875185-af96-48f6-89f8-08b30ecd26b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:02.229257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08875185-af96-48f6-89f8-08b30ecd26b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.229538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08875185-af96-48f6-89f8-08b30ecd26b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.229813 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08875185-af96-48f6-89f8-08b30ecd26b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.230575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08875185-af96-48f6-89f8-08b30ecd26b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:02.232463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08875185-af96-48f6-89f8-08b30ecd26b7 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:02.249324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:02.263232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1377/2779] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:55:02 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:02.268471 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1534475-f386-4ad3-abc9-9c887a799ac1 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:02.279282 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1534475-f386-4ad3-abc9-9c887a799ac1 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:02.336515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2bf3f-146a-4bdb-aea2-9eb3a3168dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:02.337129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2bf3f-146a-4bdb-aea2-9eb3a3168dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.337415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2bf3f-146a-4bdb-aea2-9eb3a3168dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.337698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2bf3f-146a-4bdb-aea2-9eb3a3168dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.338454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2bf3f-146a-4bdb-aea2-9eb3a3168dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:02.340432 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2bf3f-146a-4bdb-aea2-9eb3a3168dc1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:02.348568 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:02.365439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1403/2780] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:55:02 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:02.369750 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08875185-af96-48f6-89f8-08b30ecd26b7 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:02.380837 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08875185-af96-48f6-89f8-08b30ecd26b7 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:02.385527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bb5e8-e0f5-4415-8dbe-a09bc85b6235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:02.386239 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bb5e8-e0f5-4415-8dbe-a09bc85b6235 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.386525 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bb5e8-e0f5-4415-8dbe-a09bc85b6235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.386805 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bb5e8-e0f5-4415-8dbe-a09bc85b6235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.387536 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bb5e8-e0f5-4415-8dbe-a09bc85b6235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:02.389142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bb5e8-e0f5-4415-8dbe-a09bc85b6235 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:02.400657 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1378/2781] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:55:02.404829 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2bf3f-146a-4bdb-aea2-9eb3a3168dc1 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:02.415281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2bf3f-146a-4bdb-aea2-9eb3a3168dc1 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:02.420594 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb4959-162f-44ca-a113-8cbf56baa599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:02.421269 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb4959-162f-44ca-a113-8cbf56baa599 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.421554 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb4959-162f-44ca-a113-8cbf56baa599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.421849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb4959-162f-44ca-a113-8cbf56baa599 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.422783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb4959-162f-44ca-a113-8cbf56baa599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:02.424440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb4959-162f-44ca-a113-8cbf56baa599 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:02.435980 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1404/2782] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:02.440884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bb5e8-e0f5-4415-8dbe-a09bc85b6235 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:02.451913 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bb5e8-e0f5-4415-8dbe-a09bc85b6235 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:02.457283 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8f065-6ac8-49f7-90e1-d34a1fe2a7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:02.458306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8f065-6ac8-49f7-90e1-d34a1fe2a7a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.458595 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8f065-6ac8-49f7-90e1-d34a1fe2a7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.458870 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8f065-6ac8-49f7-90e1-d34a1fe2a7a2 admin admin] PATH_INFO: `/v3/projects/0cda76449b714c64b8ed3288fd2f544a/users/994d8f8eb5ab465ca2e01f88103fab73/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.474075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8f065-6ac8-49f7-90e1-d34a1fe2a7a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cda76449b714c64b8ed3288fd2f544a, user_id=994d8f8eb5ab465ca2e01f88103fab73, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:02.476764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8f065-6ac8-49f7-90e1-d34a1fe2a7a2 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:02.490516 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1379/2783] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:55:02 2022] PUT /identity/v3/projects/0cda76449b714c64b8ed3288fd2f544a/users/994d8f8eb5ab465ca2e01f88103fab73/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:02.503116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e5bee-f923-4897-8767-43bff3995926 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.503448 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e5bee-f923-4897-8767-43bff3995926 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.503727 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e5bee-f923-4897-8767-43bff3995926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.533080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-918e5bee-f923-4897-8767-43bff3995926 None None] MFA Rules not processed for user `994d8f8eb5ab465ca2e01f88103fab73`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:02.534353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-918e5bee-f923-4897-8767-43bff3995926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:02.602210 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1405/2784] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:02.606814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8f065-6ac8-49f7-90e1-d34a1fe2a7a2 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:02.671679 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8f065-6ac8-49f7-90e1-d34a1fe2a7a2 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:02.676902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34dd33-b974-4427-8d20-39bddfc47787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:02.677607 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34dd33-b974-4427-8d20-39bddfc47787 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.677891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34dd33-b974-4427-8d20-39bddfc47787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.678182 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34dd33-b974-4427-8d20-39bddfc47787 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.678874 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34dd33-b974-4427-8d20-39bddfc47787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:02.680761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34dd33-b974-4427-8d20-39bddfc47787 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:02.709196 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1380/2785] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:55:02 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:02.713557 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e5bee-f923-4897-8767-43bff3995926 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:02.725042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e5bee-f923-4897-8767-43bff3995926 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:02.785521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555ef4f-5091-4e36-bb9b-d50573c0bd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:02.786232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555ef4f-5091-4e36-bb9b-d50573c0bd9b admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.786521 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555ef4f-5091-4e36-bb9b-d50573c0bd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.786800 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555ef4f-5091-4e36-bb9b-d50573c0bd9b admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.787509 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555ef4f-5091-4e36-bb9b-d50573c0bd9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:02.789450 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555ef4f-5091-4e36-bb9b-d50573c0bd9b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:02.811567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1406/2786] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:55:02 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:02.816256 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34dd33-b974-4427-8d20-39bddfc47787 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:02.827350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34dd33-b974-4427-8d20-39bddfc47787 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:02.832076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130f6d0-02e2-4148-865b-2f580c66be47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:02.832740 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130f6d0-02e2-4148-865b-2f580c66be47 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.833018 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130f6d0-02e2-4148-865b-2f580c66be47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.833290 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130f6d0-02e2-4148-865b-2f580c66be47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.834078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130f6d0-02e2-4148-865b-2f580c66be47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:02.835663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130f6d0-02e2-4148-865b-2f580c66be47 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:02.846538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1381/2787] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:02.852527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555ef4f-5091-4e36-bb9b-d50573c0bd9b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:02.862139 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555ef4f-5091-4e36-bb9b-d50573c0bd9b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:02.866932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de966d38-09eb-4d4a-b554-1048643fc86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:02.867827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de966d38-09eb-4d4a-b554-1048643fc86a admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.868110 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de966d38-09eb-4d4a-b554-1048643fc86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.868390 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de966d38-09eb-4d4a-b554-1048643fc86a admin admin] PATH_INFO: `/v3/projects/0187bb4627274840a123853171b5647d/users/8586a2dbaffa48ac8ccb6a07148fa2f2/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.886248 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de966d38-09eb-4d4a-b554-1048643fc86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0187bb4627274840a123853171b5647d, user_id=8586a2dbaffa48ac8ccb6a07148fa2f2, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:02.889024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de966d38-09eb-4d4a-b554-1048643fc86a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:02.909118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1407/2788] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:55:02 2022] PUT /identity/v3/projects/0187bb4627274840a123853171b5647d/users/8586a2dbaffa48ac8ccb6a07148fa2f2/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:02.913375 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130f6d0-02e2-4148-865b-2f580c66be47 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:02.924931 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130f6d0-02e2-4148-865b-2f580c66be47 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:02.994494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1b75c-4543-4ffd-9ecd-5b995310cc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:02.995171 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1b75c-4543-4ffd-9ecd-5b995310cc15 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:02.995453 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1b75c-4543-4ffd-9ecd-5b995310cc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:02.995727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1b75c-4543-4ffd-9ecd-5b995310cc15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:02.996486 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1b75c-4543-4ffd-9ecd-5b995310cc15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:02.998109 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1b75c-4543-4ffd-9ecd-5b995310cc15 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:03.008285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1382/2789] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:03.012543 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de966d38-09eb-4d4a-b554-1048643fc86a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:03.024328 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de966d38-09eb-4d4a-b554-1048643fc86a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:03.029128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1056867-cd11-4d4f-85f6-280db53e1957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:03.030334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1056867-cd11-4d4f-85f6-280db53e1957 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:03.030887 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1056867-cd11-4d4f-85f6-280db53e1957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:03.031752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1056867-cd11-4d4f-85f6-280db53e1957 admin admin] PATH_INFO: `/v3/projects/0187bb4627274840a123853171b5647d/users/8586a2dbaffa48ac8ccb6a07148fa2f2/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:03.033774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1056867-cd11-4d4f-85f6-280db53e1957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0187bb4627274840a123853171b5647d, user_id=8586a2dbaffa48ac8ccb6a07148fa2f2, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:03.036459 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1056867-cd11-4d4f-85f6-280db53e1957 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:03.047633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1408/2790] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:55:03 2022] PUT /identity/v3/projects/0187bb4627274840a123853171b5647d/users/8586a2dbaffa48ac8ccb6a07148fa2f2/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:03.058329 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d1283-5d8a-4e7b-983d-43c443f3d2bb None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:03.058614 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d1283-5d8a-4e7b-983d-43c443f3d2bb None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:03.058918 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d1283-5d8a-4e7b-983d-43c443f3d2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:03.086697 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-a92d1283-5d8a-4e7b-983d-43c443f3d2bb None None] MFA Rules not processed for user `8586a2dbaffa48ac8ccb6a07148fa2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:03.087902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a92d1283-5d8a-4e7b-983d-43c443f3d2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:03.106964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:03.160691 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1383/2791] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:03.171468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1056867-cd11-4d4f-85f6-280db53e1957 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:03.173799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f1056867-cd11-4d4f-85f6-280db53e1957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:03.199756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:03.239199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1056867-cd11-4d4f-85f6-280db53e1957 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:03.242866 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9910da7-259f-40e0-b7ac-9f82358245e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:03.243522 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9910da7-259f-40e0-b7ac-9f82358245e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:03.243807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9910da7-259f-40e0-b7ac-9f82358245e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:03.244087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9910da7-259f-40e0-b7ac-9f82358245e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:03.254942 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9910da7-259f-40e0-b7ac-9f82358245e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:03.256413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9910da7-259f-40e0-b7ac-9f82358245e5 service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:03.266926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1409/2792] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:55:03 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:55:04.641688 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d1283-5d8a-4e7b-983d-43c443f3d2bb None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:04.702559 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d1283-5d8a-4e7b-983d-43c443f3d2bb None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:04.707669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:04.708544 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:04.708822 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:04.709093 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] PATH_INFO: `/v3/users/994d8f8eb5ab465ca2e01f88103fab73` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:04.710465 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=994d8f8eb5ab465ca2e01f88103fab73)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:04.712389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:04.762209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994d8f8eb5ab465ca2e01f88103fab73'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:04.762879 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994d8f8eb5ab465ca2e01f88103fab73'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:04.763513 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994d8f8eb5ab465ca2e01f88103fab73'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:04.771314 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994d8f8eb5ab465ca2e01f88103fab73'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:04.778195 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994d8f8eb5ab465ca2e01f88103fab73'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:04.795033 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1384/2793] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:55:04 2022] DELETE /identity/v3/users/994d8f8eb5ab465ca2e01f88103fab73 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:04.800219 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eb900-7513-46f8-99b3-a33e997af182 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:04.800672 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eb900-7513-46f8-99b3-a33e997af182 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:04.800793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eb900-7513-46f8-99b3-a33e997af182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:04.820203 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-384eb900-7513-46f8-99b3-a33e997af182 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:04.821408 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-384eb900-7513-46f8-99b3-a33e997af182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:04.878904 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1410/2794] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:04.888835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:04.890293 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:04.929403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:04.948695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e7865-5372-4f04-aa4e-f15e8c522e53 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:04.952376 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62406217-f152-4058-b864-980fb07d1ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:04.953034 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62406217-f152-4058-b864-980fb07d1ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:04.953324 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62406217-f152-4058-b864-980fb07d1ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:04.953598 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62406217-f152-4058-b864-980fb07d1ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:04.962426 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62406217-f152-4058-b864-980fb07d1ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:04.963848 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62406217-f152-4058-b864-980fb07d1ed3 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:04.973389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1385/2795] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:55:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:55:05.257023 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eb900-7513-46f8-99b3-a33e997af182 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:05.258566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-384eb900-7513-46f8-99b3-a33e997af182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:05.272405 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eb900-7513-46f8-99b3-a33e997af182 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:05.277774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:05.279069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:05.279331 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:05.279685 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] PATH_INFO: `/v3/projects/0cda76449b714c64b8ed3288fd2f544a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:05.280805 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cda76449b714c64b8ed3288fd2f544a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:05.283363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:05.331927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cda76449b714c64b8ed3288fd2f544a', 'request_id': 'req-bb98947d-f140-4304-a158-a47df2ea0347'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.339100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cda76449b714c64b8ed3288fd2f544a', 'request_id': 'req-bb98947d-f140-4304-a158-a47df2ea0347'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.346172 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cda76449b714c64b8ed3288fd2f544a', 'request_id': 'req-bb98947d-f140-4304-a158-a47df2ea0347'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.356591 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] The token cache is being invalidate because project 0cda76449b714c64b8ed3288fd2f544a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:55:05.356893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.357529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.359097 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1411/2796] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:55:05 2022] DELETE /identity/v3/projects/0cda76449b714c64b8ed3288fd2f544a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:05.363937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62406217-f152-4058-b864-980fb07d1ed3 service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:05.373475 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62406217-f152-4058-b864-980fb07d1ed3 service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:05.432466 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf97ce-6ae9-41bb-8926-fab5184c9e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:05.433300 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf97ce-6ae9-41bb-8926-fab5184c9e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:05.433585 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf97ce-6ae9-41bb-8926-fab5184c9e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:05.433842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf97ce-6ae9-41bb-8926-fab5184c9e4b admin admin] PATH_INFO: `/v3/users/8586a2dbaffa48ac8ccb6a07148fa2f2` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:05.435143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf97ce-6ae9-41bb-8926-fab5184c9e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8586a2dbaffa48ac8ccb6a07148fa2f2)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:05.437030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf97ce-6ae9-41bb-8926-fab5184c9e4b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:05.481632 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-34bf97ce-6ae9-41bb-8926-fab5184c9e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8586a2dbaffa48ac8ccb6a07148fa2f2'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.482301 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-34bf97ce-6ae9-41bb-8926-fab5184c9e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8586a2dbaffa48ac8ccb6a07148fa2f2'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.482898 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-34bf97ce-6ae9-41bb-8926-fab5184c9e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8586a2dbaffa48ac8ccb6a07148fa2f2'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.490526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-34bf97ce-6ae9-41bb-8926-fab5184c9e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8586a2dbaffa48ac8ccb6a07148fa2f2'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.497782 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-34bf97ce-6ae9-41bb-8926-fab5184c9e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8586a2dbaffa48ac8ccb6a07148fa2f2'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.521653 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1386/2797] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:55:05 2022] DELETE /identity/v3/users/8586a2dbaffa48ac8ccb6a07148fa2f2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:05.795541 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:05.797356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:05.813665 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98947d-f140-4304-a158-a47df2ea0347 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:05.862066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:05.877334 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e406c-7d91-40b5-9f2b-afb5bac7941c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:05.879837 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e406c-7d91-40b5-9f2b-afb5bac7941c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:05.880125 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e406c-7d91-40b5-9f2b-afb5bac7941c admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:05.880406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e406c-7d91-40b5-9f2b-afb5bac7941c admin admin] PATH_INFO: `/v3/projects/0187bb4627274840a123853171b5647d` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:05.881645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e406c-7d91-40b5-9f2b-afb5bac7941c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0187bb4627274840a123853171b5647d)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:05.883645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e406c-7d91-40b5-9f2b-afb5bac7941c admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:05.932022 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-733e406c-7d91-40b5-9f2b-afb5bac7941c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0187bb4627274840a123853171b5647d', 'request_id': 'req-733e406c-7d91-40b5-9f2b-afb5bac7941c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.940446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-733e406c-7d91-40b5-9f2b-afb5bac7941c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0187bb4627274840a123853171b5647d', 'request_id': 'req-733e406c-7d91-40b5-9f2b-afb5bac7941c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.947492 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-733e406c-7d91-40b5-9f2b-afb5bac7941c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0187bb4627274840a123853171b5647d', 'request_id': 'req-733e406c-7d91-40b5-9f2b-afb5bac7941c'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.959149 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-733e406c-7d91-40b5-9f2b-afb5bac7941c admin admin] The token cache is being invalidate because project 0187bb4627274840a123853171b5647d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:55:05.959440 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-733e406c-7d91-40b5-9f2b-afb5bac7941c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.960107 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-733e406c-7d91-40b5-9f2b-afb5bac7941c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:05.961663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1412/2798] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:55:05 2022] DELETE /identity/v3/projects/0187bb4627274840a123853171b5647d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:05.966666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b71e-2e10-4b92-9376-a4fb5ed39d91 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:05.967132 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b71e-2e10-4b92-9376-a4fb5ed39d91 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:05.967238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b71e-2e10-4b92-9376-a4fb5ed39d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:05.986905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-3d53b71e-2e10-4b92-9376-a4fb5ed39d91 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:05.988114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-3d53b71e-2e10-4b92-9376-a4fb5ed39d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:06.041306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:06.054075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1387/2799] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:06.060467 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114182f-8411-4858-a603-1269f71c7001 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.060754 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114182f-8411-4858-a603-1269f71c7001 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.061034 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114182f-8411-4858-a603-1269f71c7001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.084084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-3114182f-8411-4858-a603-1269f71c7001 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:06.085297 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3114182f-8411-4858-a603-1269f71c7001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:06.093341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1413/2800] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:06.097630 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b71e-2e10-4b92-9376-a4fb5ed39d91 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:06.107459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b71e-2e10-4b92-9376-a4fb5ed39d91 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:06.112405 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a957c0-b073-4dd4-8025-e3be9da778a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:06.113072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a957c0-b073-4dd4-8025-e3be9da778a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.113355 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a957c0-b073-4dd4-8025-e3be9da778a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.113629 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a957c0-b073-4dd4-8025-e3be9da778a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.114399 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a957c0-b073-4dd4-8025-e3be9da778a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:55:06.114706 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a957c0-b073-4dd4-8025-e3be9da778a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:06.116309 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a957c0-b073-4dd4-8025-e3be9da778a9 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:06.128934 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1388/2801] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:55:06.133330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114182f-8411-4858-a603-1269f71c7001 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:06.143605 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114182f-8411-4858-a603-1269f71c7001 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:06.148330 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bef34-1e4c-4b9b-9fc1-63206cc28e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:06.148977 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bef34-1e4c-4b9b-9fc1-63206cc28e4f admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.149257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bef34-1e4c-4b9b-9fc1-63206cc28e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.149532 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bef34-1e4c-4b9b-9fc1-63206cc28e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.150259 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bef34-1e4c-4b9b-9fc1-63206cc28e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:06.152258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bef34-1e4c-4b9b-9fc1-63206cc28e4f admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:06.181089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1414/2802] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:55:06 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:06.185265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a957c0-b073-4dd4-8025-e3be9da778a9 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:06.196372 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a957c0-b073-4dd4-8025-e3be9da778a9 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:06.253888 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44be173-22ac-4064-915b-e3d4f1eab8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:06.254503 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44be173-22ac-4064-915b-e3d4f1eab8be admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.254786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44be173-22ac-4064-915b-e3d4f1eab8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.255062 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44be173-22ac-4064-915b-e3d4f1eab8be admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.255754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44be173-22ac-4064-915b-e3d4f1eab8be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:06.257677 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44be173-22ac-4064-915b-e3d4f1eab8be admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:06.279685 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1389/2803] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:55:06 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:06.283945 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bef34-1e4c-4b9b-9fc1-63206cc28e4f admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:06.294636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bef34-1e4c-4b9b-9fc1-63206cc28e4f admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:06.299995 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ea80a-bb1e-4d26-a227-d02c264bb76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:06.300726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ea80a-bb1e-4d26-a227-d02c264bb76d admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.301015 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ea80a-bb1e-4d26-a227-d02c264bb76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.301318 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ea80a-bb1e-4d26-a227-d02c264bb76d admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.302078 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ea80a-bb1e-4d26-a227-d02c264bb76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:06.303699 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ea80a-bb1e-4d26-a227-d02c264bb76d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:06.314824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1415/2804] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:06.318859 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44be173-22ac-4064-915b-e3d4f1eab8be admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:06.329091 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44be173-22ac-4064-915b-e3d4f1eab8be admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:06.334026 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d1d36-3f7d-4ae3-898c-0ddb7ebcc6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:06.334701 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d1d36-3f7d-4ae3-898c-0ddb7ebcc6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.335006 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d1d36-3f7d-4ae3-898c-0ddb7ebcc6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.335284 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d1d36-3f7d-4ae3-898c-0ddb7ebcc6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.336013 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d1d36-3f7d-4ae3-898c-0ddb7ebcc6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:06.337603 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d1d36-3f7d-4ae3-898c-0ddb7ebcc6f9 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:06.348803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1390/2805] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:06.353375 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ea80a-bb1e-4d26-a227-d02c264bb76d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:06.364441 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ea80a-bb1e-4d26-a227-d02c264bb76d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:06.369208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70813ed-449a-40a0-9b47-476e25d49caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:06.370130 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70813ed-449a-40a0-9b47-476e25d49caf admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.370415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70813ed-449a-40a0-9b47-476e25d49caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.370693 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70813ed-449a-40a0-9b47-476e25d49caf admin admin] PATH_INFO: `/v3/projects/1004019e09954fd483c1e0591e455655/users/53399d89e12345bd95f4abcbf34ff699/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.386477 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70813ed-449a-40a0-9b47-476e25d49caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1004019e09954fd483c1e0591e455655, user_id=53399d89e12345bd95f4abcbf34ff699, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:06.389169 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70813ed-449a-40a0-9b47-476e25d49caf admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:06.402209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1416/2806] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:55:06 2022] PUT /identity/v3/projects/1004019e09954fd483c1e0591e455655/users/53399d89e12345bd95f4abcbf34ff699/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:06.471684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a359db-5221-4eee-bc90-acb4931952ee None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.472193 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a359db-5221-4eee-bc90-acb4931952ee None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.472649 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a359db-5221-4eee-bc90-acb4931952ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.507442 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-18a359db-5221-4eee-bc90-acb4931952ee None None] MFA Rules not processed for user `53399d89e12345bd95f4abcbf34ff699`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:06.508575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-18a359db-5221-4eee-bc90-acb4931952ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:06.577075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1391/2807] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:06.581790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70813ed-449a-40a0-9b47-476e25d49caf admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:06.643932 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70813ed-449a-40a0-9b47-476e25d49caf admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:06.648888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fddf4f-cc81-4b53-a37d-fb8ce0cd07c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:06.649555 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fddf4f-cc81-4b53-a37d-fb8ce0cd07c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.649833 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fddf4f-cc81-4b53-a37d-fb8ce0cd07c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.650204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fddf4f-cc81-4b53-a37d-fb8ce0cd07c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.650926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fddf4f-cc81-4b53-a37d-fb8ce0cd07c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:06.652846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fddf4f-cc81-4b53-a37d-fb8ce0cd07c4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:06.675908 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:06.690462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1417/2808] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:55:06 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:06.691916 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a359db-5221-4eee-bc90-acb4931952ee None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:06.705330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a359db-5221-4eee-bc90-acb4931952ee None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:06.755873 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:06.766769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945f088-06a4-4ab6-b279-1c844d4c5155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:06.767411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945f088-06a4-4ab6-b279-1c844d4c5155 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.767694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945f088-06a4-4ab6-b279-1c844d4c5155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.767997 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945f088-06a4-4ab6-b279-1c844d4c5155 admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.768693 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945f088-06a4-4ab6-b279-1c844d4c5155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:06.770692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945f088-06a4-4ab6-b279-1c844d4c5155 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:06.793606 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1392/2809] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:55:06 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:06.798165 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fddf4f-cc81-4b53-a37d-fb8ce0cd07c4 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:06.809338 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fddf4f-cc81-4b53-a37d-fb8ce0cd07c4 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:06.814318 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca91a64-65df-4ea3-8b81-c9b02d011e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:06.815033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca91a64-65df-4ea3-8b81-c9b02d011e0a admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.815317 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca91a64-65df-4ea3-8b81-c9b02d011e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.815645 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca91a64-65df-4ea3-8b81-c9b02d011e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.816412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca91a64-65df-4ea3-8b81-c9b02d011e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:06.818157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca91a64-65df-4ea3-8b81-c9b02d011e0a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:06.829282 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1418/2810] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:06.833443 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945f088-06a4-4ab6-b279-1c844d4c5155 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:06.843450 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945f088-06a4-4ab6-b279-1c844d4c5155 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:06.849088 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770dd4c-6ce8-4f07-a9b3-cc0060cb6c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:06.850014 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770dd4c-6ce8-4f07-a9b3-cc0060cb6c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.850298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770dd4c-6ce8-4f07-a9b3-cc0060cb6c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.850574 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770dd4c-6ce8-4f07-a9b3-cc0060cb6c45 admin admin] PATH_INFO: `/v3/projects/e64d8e86b9324c78b3ea138bd941b544/users/5ed601e75ac3414c85e414feb1ce5c71/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.868055 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770dd4c-6ce8-4f07-a9b3-cc0060cb6c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e64d8e86b9324c78b3ea138bd941b544, user_id=5ed601e75ac3414c85e414feb1ce5c71, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:06.870777 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770dd4c-6ce8-4f07-a9b3-cc0060cb6c45 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:06.883801 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1393/2811] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:55:06 2022] PUT /identity/v3/projects/e64d8e86b9324c78b3ea138bd941b544/users/5ed601e75ac3414c85e414feb1ce5c71/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:06.887937 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca91a64-65df-4ea3-8b81-c9b02d011e0a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:06.898215 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca91a64-65df-4ea3-8b81-c9b02d011e0a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:06.956242 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4021598-36e3-4dbf-bae6-9bb5f72d93ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:06.956909 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4021598-36e3-4dbf-bae6-9bb5f72d93ed admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.957196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4021598-36e3-4dbf-bae6-9bb5f72d93ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.957472 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4021598-36e3-4dbf-bae6-9bb5f72d93ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.958257 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4021598-36e3-4dbf-bae6-9bb5f72d93ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:06.959867 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4021598-36e3-4dbf-bae6-9bb5f72d93ed admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:06.970020 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1419/2812] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:06.974336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770dd4c-6ce8-4f07-a9b3-cc0060cb6c45 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:06.985228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770dd4c-6ce8-4f07-a9b3-cc0060cb6c45 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:06.990085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01533da2-5245-4426-8bf2-c4da4de989c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:06.990957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01533da2-5245-4426-8bf2-c4da4de989c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:06.991240 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01533da2-5245-4426-8bf2-c4da4de989c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:06.991517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01533da2-5245-4426-8bf2-c4da4de989c8 admin admin] PATH_INFO: `/v3/projects/e64d8e86b9324c78b3ea138bd941b544/users/5ed601e75ac3414c85e414feb1ce5c71/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:06.993488 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01533da2-5245-4426-8bf2-c4da4de989c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e64d8e86b9324c78b3ea138bd941b544, user_id=5ed601e75ac3414c85e414feb1ce5c71, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:06.996144 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01533da2-5245-4426-8bf2-c4da4de989c8 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:07.006228 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1394/2813] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:55:06 2022] PUT /identity/v3/projects/e64d8e86b9324c78b3ea138bd941b544/users/5ed601e75ac3414c85e414feb1ce5c71/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:55:07.017126 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7d0a-7cb3-4820-9285-3f04b6483a22 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:07.017348 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7d0a-7cb3-4820-9285-3f04b6483a22 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:07.017624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7d0a-7cb3-4820-9285-3f04b6483a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:07.045934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-7f7f7d0a-7cb3-4820-9285-3f04b6483a22 None None] MFA Rules not processed for user `5ed601e75ac3414c85e414feb1ce5c71`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:07.047193 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-7f7f7d0a-7cb3-4820-9285-3f04b6483a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:07.121183 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1420/2814] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:55:07.131832 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01533da2-5245-4426-8bf2-c4da4de989c8 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:07.133512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-01533da2-5245-4426-8bf2-c4da4de989c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:07.191950 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01533da2-5245-4426-8bf2-c4da4de989c8 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:07.195774 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dcce8-11fa-4b9f-8a5b-4cd657f79b11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:07.196426 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dcce8-11fa-4b9f-8a5b-4cd657f79b11 service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:07.196709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dcce8-11fa-4b9f-8a5b-4cd657f79b11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:07.196983 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dcce8-11fa-4b9f-8a5b-4cd657f79b11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:07.207686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4dcce8-11fa-4b9f-8a5b-4cd657f79b11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:07.209340 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4dcce8-11fa-4b9f-8a5b-4cd657f79b11 service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:07.220334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1395/2815] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:55:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:55:07.362864 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f7d0a-7cb3-4820-9285-3f04b6483a22 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:07.424751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f7d0a-7cb3-4820-9285-3f04b6483a22 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:07.429277 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:07.430181 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:07.430467 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:07.430741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] PATH_INFO: `/v3/users/53399d89e12345bd95f4abcbf34ff699` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:07.431926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=53399d89e12345bd95f4abcbf34ff699)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:07.433846 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:07.445511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=100 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:07.481491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53399d89e12345bd95f4abcbf34ff699'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:07.489577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53399d89e12345bd95f4abcbf34ff699'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:07.490232 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53399d89e12345bd95f4abcbf34ff699'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:07.497961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53399d89e12345bd95f4abcbf34ff699'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:07.498662 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53399d89e12345bd95f4abcbf34ff699'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:07.516784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1421/2816] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:55:07 2022] DELETE /identity/v3/users/53399d89e12345bd95f4abcbf34ff699 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:07.521463 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c701420-f9a7-47aa-a0c3-a5c51a149527 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:07.521757 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c701420-f9a7-47aa-a0c3-a5c51a149527 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:07.522061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c701420-f9a7-47aa-a0c3-a5c51a149527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:07.541814 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-4c701420-f9a7-47aa-a0c3-a5c51a149527 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:07.543045 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4c701420-f9a7-47aa-a0c3-a5c51a149527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:07.608485 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1396/2817] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:07.619613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:07.621021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:07.677749 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d4bea-648b-4f76-832c-52ac69b1ec9a admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:07.685572 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d425cd-1836-4966-8838-58d4196a046b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:07.686276 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d425cd-1836-4966-8838-58d4196a046b service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:07.686561 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d425cd-1836-4966-8838-58d4196a046b service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:07.686835 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d425cd-1836-4966-8838-58d4196a046b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:07.698327 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d425cd-1836-4966-8838-58d4196a046b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:07.698768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d425cd-1836-4966-8838-58d4196a046b service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:07.709671 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1422/2818] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:55:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:55:07.982172 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c701420-f9a7-47aa-a0c3-a5c51a149527 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:07.983762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-4c701420-f9a7-47aa-a0c3-a5c51a149527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:07.996816 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c701420-f9a7-47aa-a0c3-a5c51a149527 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:08.002521 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:08.003550 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:08.003866 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:08.004143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] PATH_INFO: `/v3/projects/1004019e09954fd483c1e0591e455655` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:08.005539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1004019e09954fd483c1e0591e455655)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:08.007835 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:08.050786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:08.054869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1004019e09954fd483c1e0591e455655', 'request_id': 'req-a1523072-a5ae-414d-8a78-dc3f4628d4a8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.062597 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1004019e09954fd483c1e0591e455655', 'request_id': 'req-a1523072-a5ae-414d-8a78-dc3f4628d4a8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.068083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1004019e09954fd483c1e0591e455655', 'request_id': 'req-a1523072-a5ae-414d-8a78-dc3f4628d4a8'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.080948 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] The token cache is being invalidate because project 1004019e09954fd483c1e0591e455655 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:55:08.081235 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.081978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.083627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1397/2819] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:55:07 2022] DELETE /identity/v3/projects/1004019e09954fd483c1e0591e455655 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:55:08.087825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d425cd-1836-4966-8838-58d4196a046b service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:08.098168 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d425cd-1836-4966-8838-58d4196a046b service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:08.157182 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c13e6-5ad2-418f-90bd-417343fe7b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:08.158111 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c13e6-5ad2-418f-90bd-417343fe7b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:08.158400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c13e6-5ad2-418f-90bd-417343fe7b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:08.158677 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c13e6-5ad2-418f-90bd-417343fe7b11 admin admin] PATH_INFO: `/v3/users/5ed601e75ac3414c85e414feb1ce5c71` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:08.159919 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c13e6-5ad2-418f-90bd-417343fe7b11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ed601e75ac3414c85e414feb1ce5c71)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:08.161851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c13e6-5ad2-418f-90bd-417343fe7b11 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:08.196538 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:08.210749 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e19c13e6-5ad2-418f-90bd-417343fe7b11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ed601e75ac3414c85e414feb1ce5c71'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.219683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e19c13e6-5ad2-418f-90bd-417343fe7b11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ed601e75ac3414c85e414feb1ce5c71'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.220539 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e19c13e6-5ad2-418f-90bd-417343fe7b11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ed601e75ac3414c85e414feb1ce5c71'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.230526 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e19c13e6-5ad2-418f-90bd-417343fe7b11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ed601e75ac3414c85e414feb1ce5c71'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.231281 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-e19c13e6-5ad2-418f-90bd-417343fe7b11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ed601e75ac3414c85e414feb1ce5c71'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.249683 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1423/2820] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:55:08 2022] DELETE /identity/v3/users/5ed601e75ac3414c85e414feb1ce5c71 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:08.521415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:08.523022 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:08.536967 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1523072-a5ae-414d-8a78-dc3f4628d4a8 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:08.594880 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fb149-a970-44cd-93fc-010207d48c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:08.595682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb149-a970-44cd-93fc-010207d48c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:08.595962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb149-a970-44cd-93fc-010207d48c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:08.596236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb149-a970-44cd-93fc-010207d48c8e admin admin] PATH_INFO: `/v3/projects/e64d8e86b9324c78b3ea138bd941b544` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:08.597411 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fb149-a970-44cd-93fc-010207d48c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e64d8e86b9324c78b3ea138bd941b544)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:08.599348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fb149-a970-44cd-93fc-010207d48c8e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:08.644365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-880fb149-a970-44cd-93fc-010207d48c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e64d8e86b9324c78b3ea138bd941b544', 'request_id': 'req-880fb149-a970-44cd-93fc-010207d48c8e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.650389 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-880fb149-a970-44cd-93fc-010207d48c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e64d8e86b9324c78b3ea138bd941b544', 'request_id': 'req-880fb149-a970-44cd-93fc-010207d48c8e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.656365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-880fb149-a970-44cd-93fc-010207d48c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e64d8e86b9324c78b3ea138bd941b544', 'request_id': 'req-880fb149-a970-44cd-93fc-010207d48c8e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.669044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-880fb149-a970-44cd-93fc-010207d48c8e admin admin] The token cache is being invalidate because project e64d8e86b9324c78b3ea138bd941b544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:55:08.669300 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-880fb149-a970-44cd-93fc-010207d48c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.670114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-880fb149-a970-44cd-93fc-010207d48c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:08.672197 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1398/2821] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:55:08 2022] DELETE /identity/v3/projects/e64d8e86b9324c78b3ea138bd941b544 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:08.677217 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fb2aa-1f21-4878-b9d1-7ff0774113d5 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:08.677507 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fb2aa-1f21-4878-b9d1-7ff0774113d5 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:08.677790 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fb2aa-1f21-4878-b9d1-7ff0774113d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:08.697724 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-638fb2aa-1f21-4878-b9d1-7ff0774113d5 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:08.698966 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-638fb2aa-1f21-4878-b9d1-7ff0774113d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:08.758274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1424/2822] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:08.766185 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f55bb5-1f42-4b6b-a368-2c15fa3d628c None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:08.766473 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f55bb5-1f42-4b6b-a368-2c15fa3d628c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:08.766745 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f55bb5-1f42-4b6b-a368-2c15fa3d628c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:08.785962 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-44f55bb5-1f42-4b6b-a368-2c15fa3d628c None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:08.787156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-44f55bb5-1f42-4b6b-a368-2c15fa3d628c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:08.794954 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1399/2823] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:08.799628 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fb2aa-1f21-4878-b9d1-7ff0774113d5 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:08.809296 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fb2aa-1f21-4878-b9d1-7ff0774113d5 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:08.814487 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc49ae-be23-4f39-8509-80b896f46bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:08.815167 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc49ae-be23-4f39-8509-80b896f46bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:08.815450 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc49ae-be23-4f39-8509-80b896f46bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:08.815726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc49ae-be23-4f39-8509-80b896f46bf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:08.816571 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc49ae-be23-4f39-8509-80b896f46bf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:55:08.816893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc49ae-be23-4f39-8509-80b896f46bf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:08.818528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc49ae-be23-4f39-8509-80b896f46bf2 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:08.830139 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1425/2824] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:55:08.834355 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f55bb5-1f42-4b6b-a368-2c15fa3d628c None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:08.843355 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f55bb5-1f42-4b6b-a368-2c15fa3d628c None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:08.848471 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79328ec3-0739-4257-a8ff-e17c9ffae39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:08.848671 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79328ec3-0739-4257-a8ff-e17c9ffae39c admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:08.848973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79328ec3-0739-4257-a8ff-e17c9ffae39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:08.849226 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79328ec3-0739-4257-a8ff-e17c9ffae39c admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:08.849953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79328ec3-0739-4257-a8ff-e17c9ffae39c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:08.851845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79328ec3-0739-4257-a8ff-e17c9ffae39c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:08.880065 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1400/2825] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:55:08 2022] POST /identity/v3/projects => generated 409 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:55:08.884219 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc49ae-be23-4f39-8509-80b896f46bf2 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:08.894431 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc49ae-be23-4f39-8509-80b896f46bf2 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:08.950456 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b64c5-3a53-44bf-ac36-7e94ffd23f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:08.951120 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b64c5-3a53-44bf-ac36-7e94ffd23f76 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:08.951404 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b64c5-3a53-44bf-ac36-7e94ffd23f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:08.951681 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b64c5-3a53-44bf-ac36-7e94ffd23f76 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:08.952363 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b64c5-3a53-44bf-ac36-7e94ffd23f76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:08.954386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b64c5-3a53-44bf-ac36-7e94ffd23f76 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:08.975141 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1426/2826] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:55:08 2022] POST /identity/v3/users => generated 305 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:08.979444 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79328ec3-0739-4257-a8ff-e17c9ffae39c admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:08.990680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79328ec3-0739-4257-a8ff-e17c9ffae39c admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:08.995294 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7745dcc-83ab-4bfa-83bb-203ff548abce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:08.995949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7745dcc-83ab-4bfa-83bb-203ff548abce admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:08.996233 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7745dcc-83ab-4bfa-83bb-203ff548abce admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:08.996509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7745dcc-83ab-4bfa-83bb-203ff548abce admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:08.997231 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7745dcc-83ab-4bfa-83bb-203ff548abce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:08.998842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7745dcc-83ab-4bfa-83bb-203ff548abce admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:09.008588 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1401/2827] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:09.012378 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b64c5-3a53-44bf-ac36-7e94ffd23f76 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:09.022373 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b64c5-3a53-44bf-ac36-7e94ffd23f76 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:09.026884 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a45f82-9281-4c43-91f1-1ef49b8f1a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:09.027528 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a45f82-9281-4c43-91f1-1ef49b8f1a0a admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:09.027824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a45f82-9281-4c43-91f1-1ef49b8f1a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:09.028084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a45f82-9281-4c43-91f1-1ef49b8f1a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:09.028802 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a45f82-9281-4c43-91f1-1ef49b8f1a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:09.030778 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a45f82-9281-4c43-91f1-1ef49b8f1a0a admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:09.041796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1427/2828] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:09.047227 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7745dcc-83ab-4bfa-83bb-203ff548abce admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:09.058245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7745dcc-83ab-4bfa-83bb-203ff548abce admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:09.062843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15ef42-6591-441b-8aec-f7a5ce57c018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:09.063708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15ef42-6591-441b-8aec-f7a5ce57c018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:09.063994 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15ef42-6591-441b-8aec-f7a5ce57c018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:09.064263 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15ef42-6591-441b-8aec-f7a5ce57c018 admin admin] PATH_INFO: `/v3/projects/4d48b8154d7540dc9ee15a0660144d87/users/abd0f8416ab34bfaa7f5d85a8418d9fa/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:09.078327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15ef42-6591-441b-8aec-f7a5ce57c018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d48b8154d7540dc9ee15a0660144d87, user_id=abd0f8416ab34bfaa7f5d85a8418d9fa, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:09.080942 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15ef42-6591-441b-8aec-f7a5ce57c018 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:09.088581 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:09.095366 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1402/2829] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:55:09 2022] PUT /identity/v3/projects/4d48b8154d7540dc9ee15a0660144d87/users/abd0f8416ab34bfaa7f5d85a8418d9fa/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:09.107540 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30b70f-8332-4e0b-8990-98a3744f0d59 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:09.108038 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30b70f-8332-4e0b-8990-98a3744f0d59 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:09.108225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30b70f-8332-4e0b-8990-98a3744f0d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:09.138155 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ff30b70f-8332-4e0b-8990-98a3744f0d59 None None] MFA Rules not processed for user `abd0f8416ab34bfaa7f5d85a8418d9fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:09.139356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ff30b70f-8332-4e0b-8990-98a3744f0d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:09.143305 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:09.211229 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1428/2830] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:09 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:09.215918 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15ef42-6591-441b-8aec-f7a5ce57c018 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:09.278661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15ef42-6591-441b-8aec-f7a5ce57c018 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:09.284179 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb1d39-75b0-4924-b467-1a60ae73db32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:09.284872 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb1d39-75b0-4924-b467-1a60ae73db32 admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:09.285156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb1d39-75b0-4924-b467-1a60ae73db32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:09.285431 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb1d39-75b0-4924-b467-1a60ae73db32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:09.286167 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb1d39-75b0-4924-b467-1a60ae73db32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:09.288064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb1d39-75b0-4924-b467-1a60ae73db32 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:09.317857 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1403/2831] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:55:09 2022] POST /identity/v3/projects => generated 407 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:09.322032 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30b70f-8332-4e0b-8990-98a3744f0d59 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:09.333824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30b70f-8332-4e0b-8990-98a3744f0d59 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:09.397768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992c2c6-dd45-4432-92c4-a3f5bdc06127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:09.398388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c2c6-dd45-4432-92c4-a3f5bdc06127 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:09.398673 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c2c6-dd45-4432-92c4-a3f5bdc06127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:09.398952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c2c6-dd45-4432-92c4-a3f5bdc06127 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:09.399660 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992c2c6-dd45-4432-92c4-a3f5bdc06127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:09.401601 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992c2c6-dd45-4432-92c4-a3f5bdc06127 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:09.423734 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1429/2832] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:55:09 2022] POST /identity/v3/users => generated 310 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:09.428712 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb1d39-75b0-4924-b467-1a60ae73db32 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:09.439407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb1d39-75b0-4924-b467-1a60ae73db32 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:09.444161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448e471-57f8-47cb-98ad-e3fe24ca4c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:09.444867 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448e471-57f8-47cb-98ad-e3fe24ca4c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:09.445078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448e471-57f8-47cb-98ad-e3fe24ca4c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:09.445350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448e471-57f8-47cb-98ad-e3fe24ca4c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:09.446160 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448e471-57f8-47cb-98ad-e3fe24ca4c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:09.447893 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448e471-57f8-47cb-98ad-e3fe24ca4c25 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:09.459383 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1404/2833] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:09.463669 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992c2c6-dd45-4432-92c4-a3f5bdc06127 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:09.474526 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992c2c6-dd45-4432-92c4-a3f5bdc06127 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:09.479129 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb523610-fd45-46c1-bc19-265d21ddc723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:09.480017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb523610-fd45-46c1-bc19-265d21ddc723 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:09.480332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb523610-fd45-46c1-bc19-265d21ddc723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:09.480607 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb523610-fd45-46c1-bc19-265d21ddc723 admin admin] PATH_INFO: `/v3/projects/0fb3dfd74f5741d48e12e41cb0004002/users/0251e088bcbc4bce9b513ac869ec7b43/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:09.496194 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb523610-fd45-46c1-bc19-265d21ddc723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb3dfd74f5741d48e12e41cb0004002, user_id=0251e088bcbc4bce9b513ac869ec7b43, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:09.498994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb523610-fd45-46c1-bc19-265d21ddc723 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:09.512742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1430/2834] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:55:09 2022] PUT /identity/v3/projects/0fb3dfd74f5741d48e12e41cb0004002/users/0251e088bcbc4bce9b513ac869ec7b43/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:09.517401 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448e471-57f8-47cb-98ad-e3fe24ca4c25 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:09.527666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448e471-57f8-47cb-98ad-e3fe24ca4c25 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:09.587075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15c64c-1528-4655-9350-0d2ff5beb24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:09.588176 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15c64c-1528-4655-9350-0d2ff5beb24a admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:09.588469 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15c64c-1528-4655-9350-0d2ff5beb24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:09.588753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15c64c-1528-4655-9350-0d2ff5beb24a admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:09.589487 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15c64c-1528-4655-9350-0d2ff5beb24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:09.591169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15c64c-1528-4655-9350-0d2ff5beb24a admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:09.604591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1405/2835] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:09.608613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb523610-fd45-46c1-bc19-265d21ddc723 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:09.619595 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb523610-fd45-46c1-bc19-265d21ddc723 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:09.624298 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6ecf4-a1c5-4653-9c95-88c3e454ee4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:09.625774 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6ecf4-a1c5-4653-9c95-88c3e454ee4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:09.626180 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6ecf4-a1c5-4653-9c95-88c3e454ee4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:09.626392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6ecf4-a1c5-4653-9c95-88c3e454ee4e admin admin] PATH_INFO: `/v3/projects/0fb3dfd74f5741d48e12e41cb0004002/users/0251e088bcbc4bce9b513ac869ec7b43/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:09.628518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6ecf4-a1c5-4653-9c95-88c3e454ee4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb3dfd74f5741d48e12e41cb0004002, user_id=0251e088bcbc4bce9b513ac869ec7b43, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:09.632072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6ecf4-a1c5-4653-9c95-88c3e454ee4e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:09.645335 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1431/2836] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:55:09 2022] PUT /identity/v3/projects/0fb3dfd74f5741d48e12e41cb0004002/users/0251e088bcbc4bce9b513ac869ec7b43/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:09.655989 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114256-beed-4b3e-91dc-e20d10a71030 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:09.656209 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114256-beed-4b3e-91dc-e20d10a71030 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:09.656523 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114256-beed-4b3e-91dc-e20d10a71030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:09.684393 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-ed114256-beed-4b3e-91dc-e20d10a71030 None None] MFA Rules not processed for user `0251e088bcbc4bce9b513ac869ec7b43`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:09.685627 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ed114256-beed-4b3e-91dc-e20d10a71030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:09.745780 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:09.757737 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1406/2837] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:09 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:09.772511 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6ecf4-a1c5-4653-9c95-88c3e454ee4e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:09.774337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-26b6ecf4-a1c5-4653-9c95-88c3e454ee4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:09.836635 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6ecf4-a1c5-4653-9c95-88c3e454ee4e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:09.840289 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b6c38-4ace-41d9-9fc5-fe5c70c5266a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:09.840935 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b6c38-4ace-41d9-9fc5-fe5c70c5266a service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:09.841213 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b6c38-4ace-41d9-9fc5-fe5c70c5266a service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:09.841486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b6c38-4ace-41d9-9fc5-fe5c70c5266a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:09.851454 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b6c38-4ace-41d9-9fc5-fe5c70c5266a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:09.852949 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b6c38-4ace-41d9-9fc5-fe5c70c5266a service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:09.858333 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:09.864134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1432/2838] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:55:09 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:55:10.020322 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed114256-beed-4b3e-91dc-e20d10a71030 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:10.030196 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed114256-beed-4b3e-91dc-e20d10a71030 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:10.035565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e07188-5dbc-474f-aac2-5c9afaa91d1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:10.035565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e07188-5dbc-474f-aac2-5c9afaa91d1a service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:10.035565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e07188-5dbc-474f-aac2-5c9afaa91d1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:10.035565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e07188-5dbc-474f-aac2-5c9afaa91d1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:10.092302 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e07188-5dbc-474f-aac2-5c9afaa91d1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:10.093753 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e07188-5dbc-474f-aac2-5c9afaa91d1a service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:10.104113 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1407/2839] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:55:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:55:19.559728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b6c38-4ace-41d9-9fc5-fe5c70c5266a service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:19.637900 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b6c38-4ace-41d9-9fc5-fe5c70c5266a service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:19.643283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:19.644157 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:19.644434 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:19.644704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] PATH_INFO: `/v3/users/abd0f8416ab34bfaa7f5d85a8418d9fa` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:19.645929 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd0f8416ab34bfaa7f5d85a8418d9fa)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:19.647998 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:19.697951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd0f8416ab34bfaa7f5d85a8418d9fa'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:19.705994 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd0f8416ab34bfaa7f5d85a8418d9fa'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:19.706700 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd0f8416ab34bfaa7f5d85a8418d9fa'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:19.713927 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd0f8416ab34bfaa7f5d85a8418d9fa'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:19.714611 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd0f8416ab34bfaa7f5d85a8418d9fa'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:19.732757 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1433/2840] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:55:19 2022] DELETE /identity/v3/users/abd0f8416ab34bfaa7f5d85a8418d9fa => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:19.737722 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042c446-6a24-4300-90d4-4e710ddc890b None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:19.738313 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042c446-6a24-4300-90d4-4e710ddc890b None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:19.738313 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042c446-6a24-4300-90d4-4e710ddc890b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:19.759050 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-e042c446-6a24-4300-90d4-4e710ddc890b None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:19.760265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e042c446-6a24-4300-90d4-4e710ddc890b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:19.819998 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1408/2841] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:19.829838 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:19.831355 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:19.895510 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19cc91-652e-406b-af3d-183c5741cee0 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:19.899361 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec2f8d-c0e0-43b9-afa8-4b08e941a954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:19.900006 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec2f8d-c0e0-43b9-afa8-4b08e941a954 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:19.900285 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec2f8d-c0e0-43b9-afa8-4b08e941a954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:19.900562 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec2f8d-c0e0-43b9-afa8-4b08e941a954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:19.910316 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec2f8d-c0e0-43b9-afa8-4b08e941a954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:19.911765 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec2f8d-c0e0-43b9-afa8-4b08e941a954 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:19.922504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1434/2842] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:55:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:55:20.205479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042c446-6a24-4300-90d4-4e710ddc890b None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:20.207126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-e042c446-6a24-4300-90d4-4e710ddc890b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:20.220258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042c446-6a24-4300-90d4-4e710ddc890b None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:20.226066 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:20.226947 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:20.227232 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:20.227506 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] PATH_INFO: `/v3/projects/4d48b8154d7540dc9ee15a0660144d87` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:20.228769 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d48b8154d7540dc9ee15a0660144d87)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:20.230770 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:20.287874 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d48b8154d7540dc9ee15a0660144d87', 'request_id': 'req-687c72f6-37d2-4a0d-8b11-902aae7d276e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.294180 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d48b8154d7540dc9ee15a0660144d87', 'request_id': 'req-687c72f6-37d2-4a0d-8b11-902aae7d276e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.303576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d48b8154d7540dc9ee15a0660144d87', 'request_id': 'req-687c72f6-37d2-4a0d-8b11-902aae7d276e'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.316619 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] The token cache is being invalidate because project 4d48b8154d7540dc9ee15a0660144d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:55:20.316909 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.317556 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.319142 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1409/2843] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:55:20 2022] DELETE /identity/v3/projects/4d48b8154d7540dc9ee15a0660144d87 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:20.323376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec2f8d-c0e0-43b9-afa8-4b08e941a954 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:20.333136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec2f8d-c0e0-43b9-afa8-4b08e941a954 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:20.351749 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:20.391633 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c7b77-c9bf-44ab-aff7-0feb67e6c775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:20.392525 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c7b77-c9bf-44ab-aff7-0feb67e6c775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:20.392832 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c7b77-c9bf-44ab-aff7-0feb67e6c775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:20.393108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c7b77-c9bf-44ab-aff7-0feb67e6c775 admin admin] PATH_INFO: `/v3/users/0251e088bcbc4bce9b513ac869ec7b43` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:20.394376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c7b77-c9bf-44ab-aff7-0feb67e6c775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0251e088bcbc4bce9b513ac869ec7b43)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:20.396353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c7b77-c9bf-44ab-aff7-0feb67e6c775 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:20.443823 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8a1c7b77-c9bf-44ab-aff7-0feb67e6c775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0251e088bcbc4bce9b513ac869ec7b43'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.451177 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8a1c7b77-c9bf-44ab-aff7-0feb67e6c775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0251e088bcbc4bce9b513ac869ec7b43'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.451793 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8a1c7b77-c9bf-44ab-aff7-0feb67e6c775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0251e088bcbc4bce9b513ac869ec7b43'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.461815 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8a1c7b77-c9bf-44ab-aff7-0feb67e6c775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0251e088bcbc4bce9b513ac869ec7b43'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.462560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8a1c7b77-c9bf-44ab-aff7-0feb67e6c775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0251e088bcbc4bce9b513ac869ec7b43'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.480254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1435/2844] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:55:20 2022] DELETE /identity/v3/users/0251e088bcbc4bce9b513ac869ec7b43 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:20.772384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:20.773972 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:20.790767 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c72f6-37d2-4a0d-8b11-902aae7d276e admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:20.804341 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:20.855744 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee2399-c11b-4511-bf2f-06ae5db43598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:20.856497 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee2399-c11b-4511-bf2f-06ae5db43598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:20.856876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee2399-c11b-4511-bf2f-06ae5db43598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:20.857133 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee2399-c11b-4511-bf2f-06ae5db43598 admin admin] PATH_INFO: `/v3/projects/0fb3dfd74f5741d48e12e41cb0004002` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:20.858578 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee2399-c11b-4511-bf2f-06ae5db43598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb3dfd74f5741d48e12e41cb0004002)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:20.860542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee2399-c11b-4511-bf2f-06ae5db43598 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:20.910524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-57ee2399-c11b-4511-bf2f-06ae5db43598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb3dfd74f5741d48e12e41cb0004002', 'request_id': 'req-57ee2399-c11b-4511-bf2f-06ae5db43598'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.917421 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-57ee2399-c11b-4511-bf2f-06ae5db43598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb3dfd74f5741d48e12e41cb0004002', 'request_id': 'req-57ee2399-c11b-4511-bf2f-06ae5db43598'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.923509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-57ee2399-c11b-4511-bf2f-06ae5db43598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb3dfd74f5741d48e12e41cb0004002', 'request_id': 'req-57ee2399-c11b-4511-bf2f-06ae5db43598'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.936287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-57ee2399-c11b-4511-bf2f-06ae5db43598 admin admin] The token cache is being invalidate because project 0fb3dfd74f5741d48e12e41cb0004002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:55:20.936827 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-57ee2399-c11b-4511-bf2f-06ae5db43598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.938019 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-57ee2399-c11b-4511-bf2f-06ae5db43598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:20.939579 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1410/2845] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:55:20 2022] DELETE /identity/v3/projects/0fb3dfd74f5741d48e12e41cb0004002 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:20.944446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b6ff3-7aae-4635-8b3b-07a2fbb23c75 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:20.944933 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b6ff3-7aae-4635-8b3b-07a2fbb23c75 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:20.945033 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b6ff3-7aae-4635-8b3b-07a2fbb23c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:20.965871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-606b6ff3-7aae-4635-8b3b-07a2fbb23c75 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:20.967222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-606b6ff3-7aae-4635-8b3b-07a2fbb23c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:21.027658 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1436/2846] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:21.037050 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427850c1-0322-4d4b-a173-bdaa9f335634 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:21.037336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427850c1-0322-4d4b-a173-bdaa9f335634 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:21.037621 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427850c1-0322-4d4b-a173-bdaa9f335634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:21.057713 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-427850c1-0322-4d4b-a173-bdaa9f335634 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:21.060606 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-427850c1-0322-4d4b-a173-bdaa9f335634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:21.067710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1411/2847] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:21.072230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b6ff3-7aae-4635-8b3b-07a2fbb23c75 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:21.082744 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b6ff3-7aae-4635-8b3b-07a2fbb23c75 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:21.087455 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc321a-6dbb-4a48-8007-26d9c22703a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:21.088118 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc321a-6dbb-4a48-8007-26d9c22703a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:21.088400 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc321a-6dbb-4a48-8007-26d9c22703a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:21.088672 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc321a-6dbb-4a48-8007-26d9c22703a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:21.089392 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc321a-6dbb-4a48-8007-26d9c22703a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:55:21.089695 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc321a-6dbb-4a48-8007-26d9c22703a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:21.091336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc321a-6dbb-4a48-8007-26d9c22703a3 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:21.103339 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1437/2848] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:55:21.108002 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427850c1-0322-4d4b-a173-bdaa9f335634 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:21.118902 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427850c1-0322-4d4b-a173-bdaa9f335634 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:21.123623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9587ef0-6490-458b-b4bb-edcb94c66bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:21.124287 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9587ef0-6490-458b-b4bb-edcb94c66bea admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:21.125085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9587ef0-6490-458b-b4bb-edcb94c66bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:21.125364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9587ef0-6490-458b-b4bb-edcb94c66bea admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:21.126086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9587ef0-6490-458b-b4bb-edcb94c66bea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:21.127982 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9587ef0-6490-458b-b4bb-edcb94c66bea admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:21.159148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1412/2849] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:55:21 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:21.163510 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc321a-6dbb-4a48-8007-26d9c22703a3 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:21.173984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc321a-6dbb-4a48-8007-26d9c22703a3 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:21.201054 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:21.235049 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5e4bc-6e20-4e69-8410-5d38b3427d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:21.235729 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5e4bc-6e20-4e69-8410-5d38b3427d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:21.236013 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5e4bc-6e20-4e69-8410-5d38b3427d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:21.236293 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5e4bc-6e20-4e69-8410-5d38b3427d58 admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:21.237250 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5e4bc-6e20-4e69-8410-5d38b3427d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:21.239209 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5e4bc-6e20-4e69-8410-5d38b3427d58 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:21.262466 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1438/2850] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:55:21 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:21.268577 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9587ef0-6490-458b-b4bb-edcb94c66bea admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:21.280009 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9587ef0-6490-458b-b4bb-edcb94c66bea admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:21.285031 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242f87f-fe9b-4ed6-a0ff-6a6bec40343a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:21.285817 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f87f-fe9b-4ed6-a0ff-6a6bec40343a admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:21.286096 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f87f-fe9b-4ed6-a0ff-6a6bec40343a admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:21.286546 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f87f-fe9b-4ed6-a0ff-6a6bec40343a admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:21.287445 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242f87f-fe9b-4ed6-a0ff-6a6bec40343a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:21.289415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242f87f-fe9b-4ed6-a0ff-6a6bec40343a admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:21.302178 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1413/2851] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:21.306381 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5e4bc-6e20-4e69-8410-5d38b3427d58 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:21.317366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5e4bc-6e20-4e69-8410-5d38b3427d58 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:21.322199 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf19de4-01a2-45b9-8293-4174dcd0edaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:21.322877 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf19de4-01a2-45b9-8293-4174dcd0edaa admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:21.323163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf19de4-01a2-45b9-8293-4174dcd0edaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:21.323442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf19de4-01a2-45b9-8293-4174dcd0edaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:21.324174 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf19de4-01a2-45b9-8293-4174dcd0edaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:21.325783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf19de4-01a2-45b9-8293-4174dcd0edaa admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:21.336771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1439/2852] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:21.341037 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242f87f-fe9b-4ed6-a0ff-6a6bec40343a admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:21.351451 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242f87f-fe9b-4ed6-a0ff-6a6bec40343a admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:21.356897 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea865cae-82e9-4a2a-99a9-2661eaed6672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:21.358517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea865cae-82e9-4a2a-99a9-2661eaed6672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:21.358517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea865cae-82e9-4a2a-99a9-2661eaed6672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:21.358783 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea865cae-82e9-4a2a-99a9-2661eaed6672 admin admin] PATH_INFO: `/v3/projects/a60cad8f82c34f3fbbb6716380810396/users/93c0ea9a8bad4843ae5eb51d3ffc1418/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:21.376330 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea865cae-82e9-4a2a-99a9-2661eaed6672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60cad8f82c34f3fbbb6716380810396, user_id=93c0ea9a8bad4843ae5eb51d3ffc1418, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:21.379047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea865cae-82e9-4a2a-99a9-2661eaed6672 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:21.395047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1414/2853] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:55:21 2022] PUT /identity/v3/projects/a60cad8f82c34f3fbbb6716380810396/users/93c0ea9a8bad4843ae5eb51d3ffc1418/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:21.405720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a3abc-28cf-47de-8c56-8ec97ecfb996 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:21.406074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a3abc-28cf-47de-8c56-8ec97ecfb996 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:21.406422 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a3abc-28cf-47de-8c56-8ec97ecfb996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:21.438174 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-052a3abc-28cf-47de-8c56-8ec97ecfb996 None None] MFA Rules not processed for user `93c0ea9a8bad4843ae5eb51d3ffc1418`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:21.439396 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-052a3abc-28cf-47de-8c56-8ec97ecfb996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:21.506422 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1440/2854] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:21.518334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea865cae-82e9-4a2a-99a9-2661eaed6672 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:21.520052 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-ea865cae-82e9-4a2a-99a9-2661eaed6672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:21.569112 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:21.589084 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea865cae-82e9-4a2a-99a9-2661eaed6672 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:21.594533 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b846d2-d90e-4bec-a960-308f2e274bae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:21.595379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b846d2-d90e-4bec-a960-308f2e274bae service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:21.595725 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b846d2-d90e-4bec-a960-308f2e274bae service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:21.595918 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b846d2-d90e-4bec-a960-308f2e274bae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:21.604993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b846d2-d90e-4bec-a960-308f2e274bae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:21.606480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b846d2-d90e-4bec-a960-308f2e274bae service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:21.617585 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1415/2855] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:55:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:55:31.503362 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:31.612348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=7 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:44.460343 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a3abc-28cf-47de-8c56-8ec97ecfb996 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:44.536612 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a3abc-28cf-47de-8c56-8ec97ecfb996 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:44.541113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:44.542072 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:44.542353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:44.542624 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] PATH_INFO: `/v3/users/93c0ea9a8bad4843ae5eb51d3ffc1418` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:44.543799 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c0ea9a8bad4843ae5eb51d3ffc1418)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:44.545807 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:44.592223 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c0ea9a8bad4843ae5eb51d3ffc1418'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:44.599405 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c0ea9a8bad4843ae5eb51d3ffc1418'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:44.600046 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c0ea9a8bad4843ae5eb51d3ffc1418'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:44.606933 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c0ea9a8bad4843ae5eb51d3ffc1418'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:44.607558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c0ea9a8bad4843ae5eb51d3ffc1418'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:44.624893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1441/2856] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:55:44 2022] DELETE /identity/v3/users/93c0ea9a8bad4843ae5eb51d3ffc1418 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:44.629456 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a941e3-1a1f-4bd7-8b0a-78f65d8eabee None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:44.630016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a941e3-1a1f-4bd7-8b0a-78f65d8eabee None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:44.630016 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a941e3-1a1f-4bd7-8b0a-78f65d8eabee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:44.649025 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-14a941e3-1a1f-4bd7-8b0a-78f65d8eabee None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:44.650265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-14a941e3-1a1f-4bd7-8b0a-78f65d8eabee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:44.711206 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1416/2857] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:44.721043 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:44.722485 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:44.786207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e72c2-eca5-48b8-9ee2-30f8e6f3e3c8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:44.789857 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4961af6-dc21-4822-8150-4bc192a717b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:44.790527 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4961af6-dc21-4822-8150-4bc192a717b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:44.790811 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4961af6-dc21-4822-8150-4bc192a717b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:44.791087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4961af6-dc21-4822-8150-4bc192a717b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:44.800545 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4961af6-dc21-4822-8150-4bc192a717b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:44.802007 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4961af6-dc21-4822-8150-4bc192a717b9 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:44.811468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1442/2858] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:55:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:55:45.079100 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a941e3-1a1f-4bd7-8b0a-78f65d8eabee None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:45.080570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-14a941e3-1a1f-4bd7-8b0a-78f65d8eabee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:45.094959 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a941e3-1a1f-4bd7-8b0a-78f65d8eabee None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:45.100346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba4328-6ff5-4614-9896-d57f45fdd439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:45.101212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba4328-6ff5-4614-9896-d57f45fdd439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:45.101498 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba4328-6ff5-4614-9896-d57f45fdd439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:45.101771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba4328-6ff5-4614-9896-d57f45fdd439 admin admin] PATH_INFO: `/v3/projects/a60cad8f82c34f3fbbb6716380810396` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:45.102984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba4328-6ff5-4614-9896-d57f45fdd439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a60cad8f82c34f3fbbb6716380810396)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:45.104964 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba4328-6ff5-4614-9896-d57f45fdd439 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:45.153040 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-18ba4328-6ff5-4614-9896-d57f45fdd439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a60cad8f82c34f3fbbb6716380810396', 'request_id': 'req-18ba4328-6ff5-4614-9896-d57f45fdd439'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:45.159735 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-18ba4328-6ff5-4614-9896-d57f45fdd439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a60cad8f82c34f3fbbb6716380810396', 'request_id': 'req-18ba4328-6ff5-4614-9896-d57f45fdd439'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:45.165128 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-18ba4328-6ff5-4614-9896-d57f45fdd439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a60cad8f82c34f3fbbb6716380810396', 'request_id': 'req-18ba4328-6ff5-4614-9896-d57f45fdd439'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:45.179620 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-18ba4328-6ff5-4614-9896-d57f45fdd439 admin admin] The token cache is being invalidate because project a60cad8f82c34f3fbbb6716380810396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:55:45.179901 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-18ba4328-6ff5-4614-9896-d57f45fdd439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:45.180538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-18ba4328-6ff5-4614-9896-d57f45fdd439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:55:45.182137 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1417/2859] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:55:45 2022] DELETE /identity/v3/projects/a60cad8f82c34f3fbbb6716380810396 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:45.187083 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9c2d0-b231-4902-93de-167b37a2ac77 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:45.187368 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9c2d0-b231-4902-93de-167b37a2ac77 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:45.187643 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9c2d0-b231-4902-93de-167b37a2ac77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:45.207415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-aff9c2d0-b231-4902-93de-167b37a2ac77 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:45.208613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-aff9c2d0-b231-4902-93de-167b37a2ac77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:45.225784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:45.270231 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1443/2860] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:55:45.279297 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd617572-63af-4a0f-bef8-ee36221f43bd None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:45.279585 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd617572-63af-4a0f-bef8-ee36221f43bd None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:45.279892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd617572-63af-4a0f-bef8-ee36221f43bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:45.299765 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-fd617572-63af-4a0f-bef8-ee36221f43bd None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:45.300959 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fd617572-63af-4a0f-bef8-ee36221f43bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:45.308754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1418/2861] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:45.312835 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9c2d0-b231-4902-93de-167b37a2ac77 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:45.322517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9c2d0-b231-4902-93de-167b37a2ac77 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:45.327018 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecdb90-4353-44cb-804e-af488c02012d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:45.327684 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecdb90-4353-44cb-804e-af488c02012d admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:45.327961 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecdb90-4353-44cb-804e-af488c02012d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:45.328233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecdb90-4353-44cb-804e-af488c02012d admin admin] PATH_INFO: `/v3/domains` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:45.329505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceecdb90-4353-44cb-804e-af488c02012d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:55:45.329808 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceecdb90-4353-44cb-804e-af488c02012d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:45.331436 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceecdb90-4353-44cb-804e-af488c02012d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:45.342780 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1444/2862] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:55:45.347346 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd617572-63af-4a0f-bef8-ee36221f43bd None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:45.356479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd617572-63af-4a0f-bef8-ee36221f43bd None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:45.361141 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727612e-02c7-4453-af54-6da07ba342da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:45.362081 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727612e-02c7-4453-af54-6da07ba342da admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:45.362364 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727612e-02c7-4453-af54-6da07ba342da admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:45.362637 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727612e-02c7-4453-af54-6da07ba342da admin admin] PATH_INFO: `/v3/projects` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:45.363327 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727612e-02c7-4453-af54-6da07ba342da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:45.365205 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727612e-02c7-4453-af54-6da07ba342da admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:45.392076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1419/2863] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:55:45 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:55:45.395971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecdb90-4353-44cb-804e-af488c02012d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:45.405771 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecdb90-4353-44cb-804e-af488c02012d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:45.463364 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70978e57-0933-40f7-869c-a3b823f7d0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:45.464027 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70978e57-0933-40f7-869c-a3b823f7d0fd admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:45.464243 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70978e57-0933-40f7-869c-a3b823f7d0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:45.464473 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70978e57-0933-40f7-869c-a3b823f7d0fd admin admin] PATH_INFO: `/v3/users` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:45.465172 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70978e57-0933-40f7-869c-a3b823f7d0fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:45.467128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70978e57-0933-40f7-869c-a3b823f7d0fd admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:45.487742 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1445/2864] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:55:45 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:55:45.491891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727612e-02c7-4453-af54-6da07ba342da admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:45.502196 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727612e-02c7-4453-af54-6da07ba342da admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:45.507122 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cb39f-5092-4547-9cba-4b6850f8d3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:45.507786 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cb39f-5092-4547-9cba-4b6850f8d3ba admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:45.508073 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cb39f-5092-4547-9cba-4b6850f8d3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:45.508354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cb39f-5092-4547-9cba-4b6850f8d3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:45.509085 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cb39f-5092-4547-9cba-4b6850f8d3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:45.510763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cb39f-5092-4547-9cba-4b6850f8d3ba admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:45.521211 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1420/2865] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:45.525228 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70978e57-0933-40f7-869c-a3b823f7d0fd admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:45.535560 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70978e57-0933-40f7-869c-a3b823f7d0fd admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:45.540175 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb323b8-70f0-4d2d-b794-435d945321c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:45.540821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb323b8-70f0-4d2d-b794-435d945321c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:45.541106 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb323b8-70f0-4d2d-b794-435d945321c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:45.541382 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb323b8-70f0-4d2d-b794-435d945321c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:45.542131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb323b8-70f0-4d2d-b794-435d945321c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:45.543721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb323b8-70f0-4d2d-b794-435d945321c1 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:45.554341 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1446/2866] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:55:45.558340 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cb39f-5092-4547-9cba-4b6850f8d3ba admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:45.568150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cb39f-5092-4547-9cba-4b6850f8d3ba admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:45.572796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bcdb0-fa38-482f-9bc8-eb6fa8d07535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:45.573662 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bcdb0-fa38-482f-9bc8-eb6fa8d07535 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:45.573979 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bcdb0-fa38-482f-9bc8-eb6fa8d07535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:45.574313 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bcdb0-fa38-482f-9bc8-eb6fa8d07535 admin admin] PATH_INFO: `/v3/projects/05d86dc172e6453bbc18f07fc07b021b/users/541d2ad56eaf44038d9566ce9832daad/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:45.589373 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bcdb0-fa38-482f-9bc8-eb6fa8d07535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d86dc172e6453bbc18f07fc07b021b, user_id=541d2ad56eaf44038d9566ce9832daad, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:45.592047 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bcdb0-fa38-482f-9bc8-eb6fa8d07535 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:45.607011 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1421/2867] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:55:45 2022] PUT /identity/v3/projects/05d86dc172e6453bbc18f07fc07b021b/users/541d2ad56eaf44038d9566ce9832daad/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:55:45.618099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ef3e0-a1d8-452e-8d7a-fc80f1d215bb None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:45.618388 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ef3e0-a1d8-452e-8d7a-fc80f1d215bb None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:45.618667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ef3e0-a1d8-452e-8d7a-fc80f1d215bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:45.647233 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-f98ef3e0-a1d8-452e-8d7a-fc80f1d215bb None None] MFA Rules not processed for user `541d2ad56eaf44038d9566ce9832daad`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:55:45.648438 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f98ef3e0-a1d8-452e-8d7a-fc80f1d215bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:45.712680 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1447/2868] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:55:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:55:45.723702 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bcdb0-fa38-482f-9bc8-eb6fa8d07535 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:45.725392 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-486bcdb0-fa38-482f-9bc8-eb6fa8d07535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:45.747407 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:45.784356 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bcdb0-fa38-482f-9bc8-eb6fa8d07535 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:45.788068 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6d012-e925-4a99-91bf-b40d5c8748ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:45.788708 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6d012-e925-4a99-91bf-b40d5c8748ea service cinder] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:45.788993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6d012-e925-4a99-91bf-b40d5c8748ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:45.789268 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6d012-e925-4a99-91bf-b40d5c8748ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:45.797447 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6d012-e925-4a99-91bf-b40d5c8748ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:45.798903 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6d012-e925-4a99-91bf-b40d5c8748ea service cinder] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:45.807596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1422/2869] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:55:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:55:48.129995 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ef3e0-a1d8-452e-8d7a-fc80f1d215bb None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:55:48.131442 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-f98ef3e0-a1d8-452e-8d7a-fc80f1d215bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:48.151001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:48.190264 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ef3e0-a1d8-452e-8d7a-fc80f1d215bb None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:55:48.194017 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b49b5-5fe4-4850-94ef-e4e399342081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:55:48.194668 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b49b5-5fe4-4850-94ef-e4e399342081 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:55:48.194952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b49b5-5fe4-4850-94ef-e4e399342081 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:55:48.195230 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b49b5-5fe4-4850-94ef-e4e399342081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:55:48.197042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1c4b49b5-5fe4-4850-94ef-e4e399342081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:55:48.206649 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b49b5-5fe4-4850-94ef-e4e399342081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:55:48.208076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b49b5-5fe4-4850-94ef-e4e399342081 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:55:48.216105 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1448/2870] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:55:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:55:55.803645 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=17 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:55:58.212427 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=18 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:56:12.229572 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6d012-e925-4a99-91bf-b40d5c8748ea service cinder] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:12.240269 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6d012-e925-4a99-91bf-b40d5c8748ea service cinder] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:12.244338 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a1780-106a-40e2-894c-0ef0c5ac41cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:12.245072 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a1780-106a-40e2-894c-0ef0c5ac41cb service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:12.245419 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a1780-106a-40e2-894c-0ef0c5ac41cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:12.245759 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a1780-106a-40e2-894c-0ef0c5ac41cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:12.247692 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-3c5a1780-106a-40e2-894c-0ef0c5ac41cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:12.259398 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a1780-106a-40e2-894c-0ef0c5ac41cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:12.260852 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a1780-106a-40e2-894c-0ef0c5ac41cb service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:12.269854 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1423/2871] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:56:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:56:12.353356 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b49b5-5fe4-4850-94ef-e4e399342081 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:12.354796 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1c4b49b5-5fe4-4850-94ef-e4e399342081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:12.366385 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b49b5-5fe4-4850-94ef-e4e399342081 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:12.370043 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2192d-8607-4cc0-82c8-7b058974e6df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:12.370704 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2192d-8607-4cc0-82c8-7b058974e6df service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:12.370988 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2192d-8607-4cc0-82c8-7b058974e6df service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:12.371245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2192d-8607-4cc0-82c8-7b058974e6df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:12.373070 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-66a2192d-8607-4cc0-82c8-7b058974e6df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:12.452536 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2192d-8607-4cc0-82c8-7b058974e6df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:12.454003 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2192d-8607-4cc0-82c8-7b058974e6df service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:12.464751 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1449/2872] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:56:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:56:22.265963 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=6 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:56:22.460066 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=34 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:56:30.756078 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a1780-106a-40e2-894c-0ef0c5ac41cb service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:30.818779 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a1780-106a-40e2-894c-0ef0c5ac41cb service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:30.824384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:30.825258 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:30.825542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:30.825815 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] PATH_INFO: `/v3/users/541d2ad56eaf44038d9566ce9832daad` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:30.827064 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] RBAC: Authorizing `identity:delete_user(user_id=541d2ad56eaf44038d9566ce9832daad)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:30.828978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:30.874110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '541d2ad56eaf44038d9566ce9832daad'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:56:30.874761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '541d2ad56eaf44038d9566ce9832daad'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:56:30.875370 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '541d2ad56eaf44038d9566ce9832daad'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:56:30.882684 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '541d2ad56eaf44038d9566ce9832daad'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:56:30.889957 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '541d2ad56eaf44038d9566ce9832daad'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:56:30.907280 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1424/2873] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:56:30 2022] DELETE /identity/v3/users/541d2ad56eaf44038d9566ce9832daad => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:56:30.911990 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6195c0-bf90-4a77-bf7c-8abbc40d6d16 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:30.912504 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6195c0-bf90-4a77-bf7c-8abbc40d6d16 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:30.912661 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6195c0-bf90-4a77-bf7c-8abbc40d6d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:30.932245 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-ae6195c0-bf90-4a77-bf7c-8abbc40d6d16 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:56:30.933469 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ae6195c0-bf90-4a77-bf7c-8abbc40d6d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:30.992110 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1450/2874] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:56:31.002762 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:31.004154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:31.060351 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0dcf5-2381-4ad9-bdfc-823473a0cdab admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:31.064163 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fa833-b8db-4b3f-8224-94a87dfc2f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:31.065362 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fa833-b8db-4b3f-8224-94a87dfc2f7f service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:31.065648 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fa833-b8db-4b3f-8224-94a87dfc2f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:31.065958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fa833-b8db-4b3f-8224-94a87dfc2f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:31.075040 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fa833-b8db-4b3f-8224-94a87dfc2f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:31.076466 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fa833-b8db-4b3f-8224-94a87dfc2f7f service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:31.086046 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1425/2875] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:56:31.388029 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6195c0-bf90-4a77-bf7c-8abbc40d6d16 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:31.389821 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-ae6195c0-bf90-4a77-bf7c-8abbc40d6d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:31.403505 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6195c0-bf90-4a77-bf7c-8abbc40d6d16 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:31.413906 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb68e5f-2659-4b6f-8090-7a844852fac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:31.414828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb68e5f-2659-4b6f-8090-7a844852fac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:31.415112 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb68e5f-2659-4b6f-8090-7a844852fac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:31.415387 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb68e5f-2659-4b6f-8090-7a844852fac4 admin admin] PATH_INFO: `/v3/projects/05d86dc172e6453bbc18f07fc07b021b` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:31.416827 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb68e5f-2659-4b6f-8090-7a844852fac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d86dc172e6453bbc18f07fc07b021b)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:31.418861 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb68e5f-2659-4b6f-8090-7a844852fac4 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:31.469266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5fb68e5f-2659-4b6f-8090-7a844852fac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d86dc172e6453bbc18f07fc07b021b', 'request_id': 'req-5fb68e5f-2659-4b6f-8090-7a844852fac4'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:56:31.477353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5fb68e5f-2659-4b6f-8090-7a844852fac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d86dc172e6453bbc18f07fc07b021b', 'request_id': 'req-5fb68e5f-2659-4b6f-8090-7a844852fac4'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:56:31.483893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5fb68e5f-2659-4b6f-8090-7a844852fac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d86dc172e6453bbc18f07fc07b021b', 'request_id': 'req-5fb68e5f-2659-4b6f-8090-7a844852fac4'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:56:31.495056 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5fb68e5f-2659-4b6f-8090-7a844852fac4 admin admin] The token cache is being invalidate because project 05d86dc172e6453bbc18f07fc07b021b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:56:31.495343 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5fb68e5f-2659-4b6f-8090-7a844852fac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:56:31.496001 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-5fb68e5f-2659-4b6f-8090-7a844852fac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:56:31.497636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1451/2876] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:56:31 2022] DELETE /identity/v3/projects/05d86dc172e6453bbc18f07fc07b021b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:56:31.502563 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282586d-13b9-4ea2-9a00-5efb2e5b5a59 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:31.502935 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282586d-13b9-4ea2-9a00-5efb2e5b5a59 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:31.503170 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282586d-13b9-4ea2-9a00-5efb2e5b5a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:31.523542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-8282586d-13b9-4ea2-9a00-5efb2e5b5a59 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:56:31.524771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8282586d-13b9-4ea2-9a00-5efb2e5b5a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:31.543526 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:56:31.594149 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1426/2877] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:56:31.603893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca20dd5-9bce-452e-91da-ece134e6c3e2 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:31.604184 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca20dd5-9bce-452e-91da-ece134e6c3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:31.604468 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca20dd5-9bce-452e-91da-ece134e6c3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:31.625446 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-bca20dd5-9bce-452e-91da-ece134e6c3e2 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:56:31.626876 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bca20dd5-9bce-452e-91da-ece134e6c3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:31.636284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1452/2878] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:56:31.640522 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282586d-13b9-4ea2-9a00-5efb2e5b5a59 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:31.650279 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282586d-13b9-4ea2-9a00-5efb2e5b5a59 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:31.655021 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2841ea-6421-482d-937f-d38296958d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:31.655672 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2841ea-6421-482d-937f-d38296958d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:31.655953 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2841ea-6421-482d-937f-d38296958d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:31.656229 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2841ea-6421-482d-937f-d38296958d14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:31.656951 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2841ea-6421-482d-937f-d38296958d14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80326) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:56:31.657260 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2841ea-6421-482d-937f-d38296958d14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:31.659420 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2841ea-6421-482d-937f-d38296958d14 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:31.670892 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1427/2879] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Jul 25 12:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:56:31.676313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca20dd5-9bce-452e-91da-ece134e6c3e2 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:31.685663 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca20dd5-9bce-452e-91da-ece134e6c3e2 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:31.690319 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f662a6-2046-4116-817f-d8146262689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:31.690969 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f662a6-2046-4116-817f-d8146262689d admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:31.691274 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f662a6-2046-4116-817f-d8146262689d admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:31.691547 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f662a6-2046-4116-817f-d8146262689d admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:31.692270 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f662a6-2046-4116-817f-d8146262689d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:31.694192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f662a6-2046-4116-817f-d8146262689d admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:31.726529 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1453/2880] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:56:31 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:56:31.731136 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2841ea-6421-482d-937f-d38296958d14 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:31.741798 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2841ea-6421-482d-937f-d38296958d14 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:31.797380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913b3b3-5c92-4e70-a8e8-b02a9e70052f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:31.798143 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913b3b3-5c92-4e70-a8e8-b02a9e70052f admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:31.798361 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913b3b3-5c92-4e70-a8e8-b02a9e70052f admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:31.798641 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913b3b3-5c92-4e70-a8e8-b02a9e70052f admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:31.799336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913b3b3-5c92-4e70-a8e8-b02a9e70052f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:31.801248 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913b3b3-5c92-4e70-a8e8-b02a9e70052f admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:31.822560 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1428/2881] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:56:31 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:56:31.827564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f662a6-2046-4116-817f-d8146262689d admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:31.838973 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f662a6-2046-4116-817f-d8146262689d admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:31.843752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c7fe7-f8e7-4fa0-a34d-f48538b566f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:31.844409 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c7fe7-f8e7-4fa0-a34d-f48538b566f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:31.844688 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c7fe7-f8e7-4fa0-a34d-f48538b566f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:31.844964 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c7fe7-f8e7-4fa0-a34d-f48538b566f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:31.846353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c7fe7-f8e7-4fa0-a34d-f48538b566f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:31.847951 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c7fe7-f8e7-4fa0-a34d-f48538b566f5 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:31.858110 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1454/2882] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:56:31.863337 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913b3b3-5c92-4e70-a8e8-b02a9e70052f admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:31.875202 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913b3b3-5c92-4e70-a8e8-b02a9e70052f admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:31.880670 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0969d16-bed7-47cc-b152-81eb23322c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:31.881448 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0969d16-bed7-47cc-b152-81eb23322c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:31.881720 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0969d16-bed7-47cc-b152-81eb23322c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:31.882131 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0969d16-bed7-47cc-b152-81eb23322c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:31.882993 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0969d16-bed7-47cc-b152-81eb23322c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:31.884830 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0969d16-bed7-47cc-b152-81eb23322c61 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:31.898404 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1429/2883] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:56:31.902872 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c7fe7-f8e7-4fa0-a34d-f48538b566f5 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:31.913578 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c7fe7-f8e7-4fa0-a34d-f48538b566f5 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:31.918706 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e2d9a-18e1-4334-86ef-c049c43072d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:31.919735 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e2d9a-18e1-4334-86ef-c049c43072d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:31.920137 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e2d9a-18e1-4334-86ef-c049c43072d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:31.920425 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e2d9a-18e1-4334-86ef-c049c43072d8 admin admin] PATH_INFO: `/v3/projects/c09181aee1194e42a4e88b4e37592e1c/users/318c288d31a74072b61ef089a028133d/roles/86e07561a0854fc28db2e9f357e53d66` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:31.935916 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e2d9a-18e1-4334-86ef-c049c43072d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09181aee1194e42a4e88b4e37592e1c, user_id=318c288d31a74072b61ef089a028133d, role_id=86e07561a0854fc28db2e9f357e53d66)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:31.938676 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e2d9a-18e1-4334-86ef-c049c43072d8 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:31.951626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1455/2884] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:56:31 2022] PUT /identity/v3/projects/c09181aee1194e42a4e88b4e37592e1c/users/318c288d31a74072b61ef089a028133d/roles/86e07561a0854fc28db2e9f357e53d66 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:56:31.958147 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e6a19-7ded-420c-8a1b-3e5727cd69c7 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:31.958549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e6a19-7ded-420c-8a1b-3e5727cd69c7 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:31.958853 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e6a19-7ded-420c-8a1b-3e5727cd69c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:31.979289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-0e9e6a19-7ded-420c-8a1b-3e5727cd69c7 None None] MFA Rules not processed for user `6bca01c7266944329ecb3cbe0887bc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:56:31.980512 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0e9e6a19-7ded-420c-8a1b-3e5727cd69c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:32.038549 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1430/2885] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:56:32.048515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e2d9a-18e1-4334-86ef-c049c43072d8 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:32.050312 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-1a5e2d9a-18e1-4334-86ef-c049c43072d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:32.073088 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:56:32.109406 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e2d9a-18e1-4334-86ef-c049c43072d8 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:32.113413 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a595eb-19f9-4dce-8ea5-7b665c1db440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:32.114085 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a595eb-19f9-4dce-8ea5-7b665c1db440 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:32.114367 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a595eb-19f9-4dce-8ea5-7b665c1db440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:32.114640 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a595eb-19f9-4dce-8ea5-7b665c1db440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:32.124756 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a595eb-19f9-4dce-8ea5-7b665c1db440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:32.126239 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a595eb-19f9-4dce-8ea5-7b665c1db440 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:32.136379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1456/2886] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:56:36.638156 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1a793-41d9-449d-b74c-3ae5875b0e00 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:36.638658 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1a793-41d9-449d-b74c-3ae5875b0e00 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:36.638754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1a793-41d9-449d-b74c-3ae5875b0e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:36.670842 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-56e1a793-41d9-449d-b74c-3ae5875b0e00 None None] MFA Rules not processed for user `318c288d31a74072b61ef089a028133d`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:56:36.672009 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-56e1a793-41d9-449d-b74c-3ae5875b0e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:36.694800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:56:36.753218 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1431/2887] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:56:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:56:36.757819 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a595eb-19f9-4dce-8ea5-7b665c1db440 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:36.768222 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a595eb-19f9-4dce-8ea5-7b665c1db440 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:36.773084 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5462e-76c5-48ee-bd1a-97f836d2e8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:36.773739 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5462e-76c5-48ee-bd1a-97f836d2e8ea admin admin] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:36.774076 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5462e-76c5-48ee-bd1a-97f836d2e8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:36.774376 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5462e-76c5-48ee-bd1a-97f836d2e8ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:36.775092 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5462e-76c5-48ee-bd1a-97f836d2e8ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:36.777021 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5462e-76c5-48ee-bd1a-97f836d2e8ea admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:36.807383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1457/2888] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Jul 25 12:56:36 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:56:36.811265 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1a793-41d9-449d-b74c-3ae5875b0e00 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:36.825350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1a793-41d9-449d-b74c-3ae5875b0e00 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:36.887869 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74502165-e2a1-422e-bb4a-f8e6a975d71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:36.888553 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74502165-e2a1-422e-bb4a-f8e6a975d71b admin admin] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:36.888843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74502165-e2a1-422e-bb4a-f8e6a975d71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:36.889117 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74502165-e2a1-422e-bb4a-f8e6a975d71b admin admin] PATH_INFO: `/v3/users` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:36.889812 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74502165-e2a1-422e-bb4a-f8e6a975d71b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:36.891764 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74502165-e2a1-422e-bb4a-f8e6a975d71b admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:36.917254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1432/2889] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Jul 25 12:56:36 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:56:36.918636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5462e-76c5-48ee-bd1a-97f836d2e8ea admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:36.931602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5462e-76c5-48ee-bd1a-97f836d2e8ea admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:36.936768 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd71da-a375-46d8-88ca-c39c31178320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:36.937475 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd71da-a375-46d8-88ca-c39c31178320 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:36.938439 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd71da-a375-46d8-88ca-c39c31178320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:36.938721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd71da-a375-46d8-88ca-c39c31178320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:36.939493 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd71da-a375-46d8-88ca-c39c31178320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:36.941187 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd71da-a375-46d8-88ca-c39c31178320 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:36.952894 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1458/2890] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:56:36.957694 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74502165-e2a1-422e-bb4a-f8e6a975d71b admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:36.968067 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74502165-e2a1-422e-bb4a-f8e6a975d71b admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:36.973710 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf578e-1f3f-46cc-bae6-9dc070d1ea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:36.974682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf578e-1f3f-46cc-bae6-9dc070d1ea93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:36.974975 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf578e-1f3f-46cc-bae6-9dc070d1ea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:36.975238 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf578e-1f3f-46cc-bae6-9dc070d1ea93 admin admin] PATH_INFO: `/v3/projects/02eb52c6faa844d6a087c371b796c04e/users/134a324d31944b3a8fe7f9d3fb58a09c/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:36.992319 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf578e-1f3f-46cc-bae6-9dc070d1ea93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02eb52c6faa844d6a087c371b796c04e, user_id=134a324d31944b3a8fe7f9d3fb58a09c, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:36.995061 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf578e-1f3f-46cc-bae6-9dc070d1ea93 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:37.009041 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1433/2891] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:56:36 2022] PUT /identity/v3/projects/02eb52c6faa844d6a087c371b796c04e/users/134a324d31944b3a8fe7f9d3fb58a09c/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:56:37.013730 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd71da-a375-46d8-88ca-c39c31178320 admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:37.028712 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd71da-a375-46d8-88ca-c39c31178320 admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:37.114079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e29ae-cba5-4e06-9845-abcd8922af45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:37.114759 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e29ae-cba5-4e06-9845-abcd8922af45 admin admin] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:37.115042 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e29ae-cba5-4e06-9845-abcd8922af45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:37.115320 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e29ae-cba5-4e06-9845-abcd8922af45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:37.116058 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e29ae-cba5-4e06-9845-abcd8922af45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:37.118074 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e29ae-cba5-4e06-9845-abcd8922af45 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:37.128486 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1459/2892] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Jul 25 12:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:56:37.132785 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf578e-1f3f-46cc-bae6-9dc070d1ea93 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:37.144576 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf578e-1f3f-46cc-bae6-9dc070d1ea93 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:37.149635 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06583e5-f20b-4573-9a68-cd955e5c9f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:37.150695 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06583e5-f20b-4573-9a68-cd955e5c9f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:37.150981 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06583e5-f20b-4573-9a68-cd955e5c9f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:37.151257 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06583e5-f20b-4573-9a68-cd955e5c9f19 admin admin] PATH_INFO: `/v3/projects/02eb52c6faa844d6a087c371b796c04e/users/134a324d31944b3a8fe7f9d3fb58a09c/roles/dc72adb78cf8456d9f0a66875d92c83a` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:37.153350 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06583e5-f20b-4573-9a68-cd955e5c9f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02eb52c6faa844d6a087c371b796c04e, user_id=134a324d31944b3a8fe7f9d3fb58a09c, role_id=dc72adb78cf8456d9f0a66875d92c83a)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:37.156057 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06583e5-f20b-4573-9a68-cd955e5c9f19 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:37.168024 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1434/2893] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Jul 25 12:56:37 2022] PUT /identity/v3/projects/02eb52c6faa844d6a087c371b796c04e/users/134a324d31944b3a8fe7f9d3fb58a09c/roles/dc72adb78cf8456d9f0a66875d92c83a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:56:41.704123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663f993-8f19-4e1f-a2cd-a5557964e5e3 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:41.705447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663f993-8f19-4e1f-a2cd-a5557964e5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:41.705447 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663f993-8f19-4e1f-a2cd-a5557964e5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:41.741135 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-9663f993-8f19-4e1f-a2cd-a5557964e5e3 None None] MFA Rules not processed for user `134a324d31944b3a8fe7f9d3fb58a09c`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:56:41.742425 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-9663f993-8f19-4e1f-a2cd-a5557964e5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:41.833087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1460/2894] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:56:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:56:41.847354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06583e5-f20b-4573-9a68-cd955e5c9f19 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:56:41.850023 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-f06583e5-f20b-4573-9a68-cd955e5c9f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:56:41.941320 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06583e5-f20b-4573-9a68-cd955e5c9f19 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:56:41.949738 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc142c5f-63a4-4af5-8e98-9a9f5940ffd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:56:41.950616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc142c5f-63a4-4af5-8e98-9a9f5940ffd2 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:56:41.950984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc142c5f-63a4-4af5-8e98-9a9f5940ffd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:56:41.951446 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc142c5f-63a4-4af5-8e98-9a9f5940ffd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:56:41.959639 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:56:42.025221 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc142c5f-63a4-4af5-8e98-9a9f5940ffd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:56:42.026802 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc142c5f-63a4-4af5-8e98-9a9f5940ffd2 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:56:42.036922 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1435/2895] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:56:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:56:51.821079 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:56:52.032432 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=24 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:57:06.945571 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6296812-d179-4419-b946-57904b519033 None None] REQUEST_METHOD: `POST` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:57:06.946154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6296812-d179-4419-b946-57904b519033 None None] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:57:06.946154 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6296812-d179-4419-b946-57904b519033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:57:06.978467 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.auth.core [None req-e6296812-d179-4419-b946-57904b519033 None None] MFA Rules not processed for user `318c288d31a74072b61ef089a028133d`. Rule list: `[]` (Enabled: `True`). {{(pid=80327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:57:06.980386 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e6296812-d179-4419-b946-57904b519033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:57:06.987930 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1461/2896] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:57:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:57:07.335075 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc142c5f-63a4-4af5-8e98-9a9f5940ffd2 service nova] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:57:07.389057 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc142c5f-63a4-4af5-8e98-9a9f5940ffd2 service nova] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:57:07.393298 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568223b0-83f0-40c4-91f4-4c248f9a448a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:57:07.393956 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568223b0-83f0-40c4-91f4-4c248f9a448a service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:57:07.394247 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568223b0-83f0-40c4-91f4-4c248f9a448a service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:57:07.394524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568223b0-83f0-40c4-91f4-4c248f9a448a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:57:07.396390 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-568223b0-83f0-40c4-91f4-4c248f9a448a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:57:07.406048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568223b0-83f0-40c4-91f4-4c248f9a448a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:57:07.407493 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568223b0-83f0-40c4-91f4-4c248f9a448a service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:57:07.416123 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1436/2897] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:57:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:57:07.865196 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6296812-d179-4419-b946-57904b519033 None None] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:57:07.867262 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e6296812-d179-4419-b946-57904b519033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:57:07.945741 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6296812-d179-4419-b946-57904b519033 None None] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:57:07.951266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aec7a4-aa5e-4f74-af66-f6d07ee029bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:57:07.951926 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aec7a4-aa5e-4f74-af66-f6d07ee029bd service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:57:07.952212 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aec7a4-aa5e-4f74-af66-f6d07ee029bd service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:57:07.952488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aec7a4-aa5e-4f74-af66-f6d07ee029bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:57:07.954336 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-34aec7a4-aa5e-4f74-af66-f6d07ee029bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:57:07.966437 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aec7a4-aa5e-4f74-af66-f6d07ee029bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:57:07.968024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aec7a4-aa5e-4f74-af66-f6d07ee029bd service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:57:07.978353 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1462/2898] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:57:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:57:10.810161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568223b0-83f0-40c4-91f4-4c248f9a448a service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:57:10.812519 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-568223b0-83f0-40c4-91f4-4c248f9a448a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:57:10.833582 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568223b0-83f0-40c4-91f4-4c248f9a448a service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:57:10.840478 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb662282-faf8-4387-b9e8-5a3bfcce9a99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:57:10.841254 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb662282-faf8-4387-b9e8-5a3bfcce9a99 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:57:10.841846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb662282-faf8-4387-b9e8-5a3bfcce9a99 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:57:10.842378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb662282-faf8-4387-b9e8-5a3bfcce9a99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:57:10.844575 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fb662282-faf8-4387-b9e8-5a3bfcce9a99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:57:10.858387 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb662282-faf8-4387-b9e8-5a3bfcce9a99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:57:10.860846 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb662282-faf8-4387-b9e8-5a3bfcce9a99 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:57:10.874053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1437/2899] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:57:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:57:17.973902 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=35 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:57:20.870225 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=32 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:57:21.437513 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aec7a4-aa5e-4f74-af66-f6d07ee029bd service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:57:21.451998 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aec7a4-aa5e-4f74-af66-f6d07ee029bd service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:57:21.458444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d83c8c-21bc-41af-94bd-5466051a1812 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:57:21.459208 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d83c8c-21bc-41af-94bd-5466051a1812 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:57:21.459488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d83c8c-21bc-41af-94bd-5466051a1812 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:57:21.459767 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d83c8c-21bc-41af-94bd-5466051a1812 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:57:21.461784 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-17d83c8c-21bc-41af-94bd-5466051a1812 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:57:21.475136 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d83c8c-21bc-41af-94bd-5466051a1812 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:57:21.476728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d83c8c-21bc-41af-94bd-5466051a1812 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:57:21.488488 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1463/2900] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:57:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:57:24.630771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb662282-faf8-4387-b9e8-5a3bfcce9a99 service nova] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:57:24.632245 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-fb662282-faf8-4387-b9e8-5a3bfcce9a99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:57:24.645015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb662282-faf8-4387-b9e8-5a3bfcce9a99 service nova] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:57:24.648623 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac42c0-c56f-4115-bab8-752e0aa880ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:57:24.649336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac42c0-c56f-4115-bab8-752e0aa880ed service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:57:24.649729 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac42c0-c56f-4115-bab8-752e0aa880ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:57:24.650104 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac42c0-c56f-4115-bab8-752e0aa880ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:57:24.660796 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac42c0-c56f-4115-bab8-752e0aa880ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:57:24.662289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac42c0-c56f-4115-bab8-752e0aa880ed service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:57:24.673043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1438/2901] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:57:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:57:31.483186 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=6 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:57:34.668128 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=6 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:58:09.227423 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d83c8c-21bc-41af-94bd-5466051a1812 service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:58:09.246919 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d83c8c-21bc-41af-94bd-5466051a1812 service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:58:09.250412 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b651bc3-cfc4-4948-bbd3-4dd105436e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:58:09.251533 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b651bc3-cfc4-4948-bbd3-4dd105436e73 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:58:09.251812 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b651bc3-cfc4-4948-bbd3-4dd105436e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:58:09.252080 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b651bc3-cfc4-4948-bbd3-4dd105436e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:58:09.253824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-0b651bc3-cfc4-4948-bbd3-4dd105436e73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:58:09.334491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b651bc3-cfc4-4948-bbd3-4dd105436e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:58:09.335934 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b651bc3-cfc4-4948-bbd3-4dd105436e73 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:58:09.344836 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1464/2902] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:58:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:58:14.146865 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac42c0-c56f-4115-bab8-752e0aa880ed service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:58:14.165581 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac42c0-c56f-4115-bab8-752e0aa880ed service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:58:14.169241 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb909f-b2af-4ceb-97aa-9bcc7d6535be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:58:14.169886 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb909f-b2af-4ceb-97aa-9bcc7d6535be service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:58:14.170198 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb909f-b2af-4ceb-97aa-9bcc7d6535be service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:58:14.170539 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb909f-b2af-4ceb-97aa-9bcc7d6535be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:58:14.172266 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-23cb909f-b2af-4ceb-97aa-9bcc7d6535be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:58:14.262480 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb909f-b2af-4ceb-97aa-9bcc7d6535be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:58:14.263839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb909f-b2af-4ceb-97aa-9bcc7d6535be service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:58:14.282352 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1439/2903] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:58:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:58:17.325698 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b651bc3-cfc4-4948-bbd3-4dd105436e73 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:58:17.337667 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b651bc3-cfc4-4948-bbd3-4dd105436e73 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:58:17.343100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49785346-76ab-4e6f-a43a-faecadeff6e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:58:17.343984 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49785346-76ab-4e6f-a43a-faecadeff6e5 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:58:17.344266 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49785346-76ab-4e6f-a43a-faecadeff6e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:58:17.344567 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49785346-76ab-4e6f-a43a-faecadeff6e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:58:17.346415 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-49785346-76ab-4e6f-a43a-faecadeff6e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:58:17.377018 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49785346-76ab-4e6f-a43a-faecadeff6e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:58:17.378672 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49785346-76ab-4e6f-a43a-faecadeff6e5 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:58:17.395194 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1465/2904] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:58:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:58:20.543542 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb909f-b2af-4ceb-97aa-9bcc7d6535be service neutron] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:58:20.565418 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb909f-b2af-4ceb-97aa-9bcc7d6535be service neutron] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:58:20.572087 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de10cd5-240e-425c-a257-04dabcce0f80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:58:20.573742 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de10cd5-240e-425c-a257-04dabcce0f80 service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:58:20.575570 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de10cd5-240e-425c-a257-04dabcce0f80 service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:58:20.576110 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de10cd5-240e-425c-a257-04dabcce0f80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:58:20.582943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-8de10cd5-240e-425c-a257-04dabcce0f80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:58:20.601591 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de10cd5-240e-425c-a257-04dabcce0f80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:58:20.601789 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49785346-76ab-4e6f-a43a-faecadeff6e5 service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:58:20.603306 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-49785346-76ab-4e6f-a43a-faecadeff6e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:58:20.604514 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de10cd5-240e-425c-a257-04dabcce0f80 service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:58:20.631709 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49785346-76ab-4e6f-a43a-faecadeff6e5 service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:58:20.634524 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1440/2905] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:58:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:58:20.637720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:58:20.638644 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:58:20.638918 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:58:20.639189 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] PATH_INFO: `/v3/users/42b64d8fffbc4a0fa171099d5db580a0` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:58:20.641999 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b64d8fffbc4a0fa171099d5db580a0)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:58:20.644589 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:58:20.707912 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b64d8fffbc4a0fa171099d5db580a0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:20.715251 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b64d8fffbc4a0fa171099d5db580a0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:20.715982 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b64d8fffbc4a0fa171099d5db580a0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:20.723847 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b64d8fffbc4a0fa171099d5db580a0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:20.724559 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b64d8fffbc4a0fa171099d5db580a0'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:20.744123 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1466/2906] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:58:20 2022] DELETE /identity/v3/users/42b64d8fffbc4a0fa171099d5db580a0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:58:20.933295 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de10cd5-240e-425c-a257-04dabcce0f80 service placement] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:58:20.949702 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de10cd5-240e-425c-a257-04dabcce0f80 service placement] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:58:21.009538 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:58:21.010666 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:58:21.010949 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:58:21.011223 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] PATH_INFO: `/v3/projects/b7a69b02cd124acdb14727ea0711c1f8` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:58:21.012415 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7a69b02cd124acdb14727ea0711c1f8)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:58:21.014354 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:58:21.059289 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:58:21.070000 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7a69b02cd124acdb14727ea0711c1f8', 'request_id': 'req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.076306 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7a69b02cd124acdb14727ea0711c1f8', 'request_id': 'req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.082076 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7a69b02cd124acdb14727ea0711c1f8', 'request_id': 'req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.095479 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] The token cache is being invalidate because project b7a69b02cd124acdb14727ea0711c1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:58:21.095761 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.096409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.097984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1441/2907] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:58:20 2022] DELETE /identity/v3/projects/b7a69b02cd124acdb14727ea0711c1f8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:58:21.102100 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:58:21.103614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:58:21.118592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c081f-f95c-48d6-8106-aec3e5c2d20b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:58:21.135379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:58:21.185820 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:58:21.186753 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:58:21.187104 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:58:21.188089 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] PATH_INFO: `/v3/users/6059757c96c2499ba82ef0d39a1a6355` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:58:21.188549 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6059757c96c2499ba82ef0d39a1a6355)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:58:21.190515 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:58:21.241637 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6059757c96c2499ba82ef0d39a1a6355'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.249601 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6059757c96c2499ba82ef0d39a1a6355'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.250260 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6059757c96c2499ba82ef0d39a1a6355'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.257366 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6059757c96c2499ba82ef0d39a1a6355'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.258036 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6059757c96c2499ba82ef0d39a1a6355'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.274558 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1467/2908] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 12:58:21 2022] DELETE /identity/v3/users/6059757c96c2499ba82ef0d39a1a6355 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:58:21.457543 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:58:21.459212 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:58:21.473565 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa4ca6-7d96-4ee3-b2cf-446b4f31c036 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:58:21.531699 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:58:21.533092 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:58:21.533379 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:58:21.533656 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] PATH_INFO: `/v3/projects/148c9c2c71984436abc9f275e4f890c3` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:58:21.534876 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] RBAC: Authorizing `identity:delete_project(project_id=148c9c2c71984436abc9f275e4f890c3)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:58:21.536773 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:58:21.583958 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '148c9c2c71984436abc9f275e4f890c3', 'request_id': 'req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.590054 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '148c9c2c71984436abc9f275e4f890c3', 'request_id': 'req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.596050 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '148c9c2c71984436abc9f275e4f890c3', 'request_id': 'req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.608126 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] The token cache is being invalidate because project 148c9c2c71984436abc9f275e4f890c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80326) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:58:21.608409 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.609050 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:58:21.610615 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1442/2909] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 12:58:21 2022] DELETE /identity/v3/projects/148c9c2c71984436abc9f275e4f890c3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:58:31.271204 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=46,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:58:31.605056 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:58:46.429828 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:58:46.431313 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:58:46.521877 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf202fff-0c19-437a-aa8a-3008b14dfd6b admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:58:46.582491 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cb0f5-316a-414e-ae52-c15304543c03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:58:46.583156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cb0f5-316a-414e-ae52-c15304543c03 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:58:46.583451 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cb0f5-316a-414e-ae52-c15304543c03 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:58:46.583760 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cb0f5-316a-414e-ae52-c15304543c03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:58:46.585542 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5b6cb0f5-316a-414e-ae52-c15304543c03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:58:46.597636 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6cb0f5-316a-414e-ae52-c15304543c03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:58:46.599256 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6cb0f5-316a-414e-ae52-c15304543c03 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:58:46.609221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1468/2910] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:58:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:58:56.604889 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=67 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:59:53.556098 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:59:53.557943 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:59:53.635590 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c7b25-55f4-4f05-a19e-1a07d6318adb admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:59:53.639154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258e8ee-0066-4520-86a3-97d1abf88442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:59:53.639799 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258e8ee-0066-4520-86a3-97d1abf88442 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:59:53.640105 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258e8ee-0066-4520-86a3-97d1abf88442 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:59:53.640384 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258e8ee-0066-4520-86a3-97d1abf88442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:59:53.642202 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2258e8ee-0066-4520-86a3-97d1abf88442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:59:53.736899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258e8ee-0066-4520-86a3-97d1abf88442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:59:53.738374 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258e8ee-0066-4520-86a3-97d1abf88442 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:59:53.760613 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1443/2911] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:59:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:59:54.941851 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cb0f5-316a-414e-ae52-c15304543c03 service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:59:54.943393 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-5b6cb0f5-316a-414e-ae52-c15304543c03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:59:54.955383 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cb0f5-316a-414e-ae52-c15304543c03 service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:59:54.958971 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f094a-f6f3-4419-b414-25274a6c6309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:59:54.959605 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f094a-f6f3-4419-b414-25274a6c6309 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:59:54.959888 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f094a-f6f3-4419-b414-25274a6c6309 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:59:54.960163 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f094a-f6f3-4419-b414-25274a6c6309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:59:54.961860 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-849f094a-f6f3-4419-b414-25274a6c6309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:59:54.976570 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f094a-f6f3-4419-b414-25274a6c6309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:59:54.978069 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f094a-f6f3-4419-b414-25274a6c6309 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:59:54.989517 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1469/2912] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:59:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:59:57.269161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809874c-e5ce-4da6-be3d-6df4f081497c None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:59:57.269864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809874c-e5ce-4da6-be3d-6df4f081497c None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:59:57.269864 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809874c-e5ce-4da6-be3d-6df4f081497c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:59:57.289282 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-3809874c-e5ce-4da6-be3d-6df4f081497c None None] MFA Rules not processed for user `318c288d31a74072b61ef089a028133d`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:59:57.290634 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-3809874c-e5ce-4da6-be3d-6df4f081497c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:59:57.341800 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=100 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:59:57.349973 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1444/2913] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 12:59:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:59:57.666669 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f094a-f6f3-4419-b414-25274a6c6309 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:59:57.675192 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f094a-f6f3-4419-b414-25274a6c6309 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:59:57.678814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ef3df-da3e-4e6a-aad2-15cfb1684582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:59:57.679444 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ef3df-da3e-4e6a-aad2-15cfb1684582 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:59:57.679721 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ef3df-da3e-4e6a-aad2-15cfb1684582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:59:57.679997 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ef3df-da3e-4e6a-aad2-15cfb1684582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:59:57.681732 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3a7ef3df-da3e-4e6a-aad2-15cfb1684582 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:59:57.691726 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ef3df-da3e-4e6a-aad2-15cfb1684582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:59:57.693134 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ef3df-da3e-4e6a-aad2-15cfb1684582 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:59:57.701752 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1470/2914] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 12:59:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:00:07.697664 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=12 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 13:00:53.115030 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809874c-e5ce-4da6-be3d-6df4f081497c None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:00:53.124938 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809874c-e5ce-4da6-be3d-6df4f081497c None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:00:53.129709 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54235ce9-5ce7-48a3-831b-ca4001271667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:00:53.130686 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54235ce9-5ce7-48a3-831b-ca4001271667 service neutron] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:00:53.131751 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54235ce9-5ce7-48a3-831b-ca4001271667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:00:53.132007 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54235ce9-5ce7-48a3-831b-ca4001271667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:00:53.133905 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-54235ce9-5ce7-48a3-831b-ca4001271667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:00:53.144169 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54235ce9-5ce7-48a3-831b-ca4001271667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:00:53.145596 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54235ce9-5ce7-48a3-831b-ca4001271667 service neutron] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:00:53.154403 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1445/2915] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 13:00:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:01:03.150259 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=6 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 13:02:19.132283 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ef3df-da3e-4e6a-aad2-15cfb1684582 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:02:19.133687 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-3a7ef3df-da3e-4e6a-aad2-15cfb1684582 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:02:19.213071 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ef3df-da3e-4e6a-aad2-15cfb1684582 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:02:19.216705 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9a66e-e617-4884-8cf8-51ee93214c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb4c5027951c4c55bd826943cec15309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:02:19.217347 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9a66e-e617-4884-8cf8-51ee93214c5e service cinder] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:02:19.218305 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9a66e-e617-4884-8cf8-51ee93214c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:02:19.218577 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9a66e-e617-4884-8cf8-51ee93214c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:02:19.220391 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-bcd9a66e-e617-4884-8cf8-51ee93214c5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:02:19.231372 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9a66e-e617-4884-8cf8-51ee93214c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:02:19.232792 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9a66e-e617-4884-8cf8-51ee93214c5e service cinder] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:02:19.241869 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1471/2916] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 13:02:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:02:29.237544 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=35 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 13:02:30.245484 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8c2b9-2f76-48e5-8f42-6c14d5604c10 None None] REQUEST_METHOD: `POST` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:02:30.246039 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8c2b9-2f76-48e5-8f42-6c14d5604c10 None None] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:02:30.246145 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8c2b9-2f76-48e5-8f42-6c14d5604c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:02:30.264599 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.auth.core [None req-c0a8c2b9-2f76-48e5-8f42-6c14d5604c10 None None] MFA Rules not processed for user `318c288d31a74072b61ef089a028133d`. Rule list: `[]` (Enabled: `True`). {{(pid=80326) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 13:02:30.265804 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c0a8c2b9-2f76-48e5-8f42-6c14d5604c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:02:30.273148 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1446/2917] 10.222.0.60 () {58 vars in 958 bytes} [Mon Jul 25 13:02:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 13:02:30.584720 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9a66e-e617-4884-8cf8-51ee93214c5e service cinder] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:02:30.594108 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9a66e-e617-4884-8cf8-51ee93214c5e service cinder] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:02:30.597613 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205f274-e26d-4f94-a642-f1d1f4ca2d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:02:30.598284 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205f274-e26d-4f94-a642-f1d1f4ca2d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:02:30.598566 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205f274-e26d-4f94-a642-f1d1f4ca2d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:02:30.598840 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205f274-e26d-4f94-a642-f1d1f4ca2d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:02:30.600602 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4205f274-e26d-4f94-a642-f1d1f4ca2d97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:02:30.610064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205f274-e26d-4f94-a642-f1d1f4ca2d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:02:30.611482 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205f274-e26d-4f94-a642-f1d1f4ca2d97 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:02:30.619722 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1472/2918] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 13:02:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:02:31.064446 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8c2b9-2f76-48e5-8f42-6c14d5604c10 None None] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:02:31.066048 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-c0a8c2b9-2f76-48e5-8f42-6c14d5604c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:02:31.081336 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8c2b9-2f76-48e5-8f42-6c14d5604c10 None None] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:02:31.086727 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deea5a-0299-434c-9937-0c029176a4b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:02:31.087378 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deea5a-0299-434c-9937-0c029176a4b7 service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:02:31.087661 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deea5a-0299-434c-9937-0c029176a4b7 service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:02:31.087936 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deea5a-0299-434c-9937-0c029176a4b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:02:31.089715 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-a4deea5a-0299-434c-9937-0c029176a4b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:02:31.101459 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4deea5a-0299-434c-9937-0c029176a4b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:02:31.103077 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4deea5a-0299-434c-9937-0c029176a4b7 service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:02:31.113803 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1447/2919] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 13:02:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:02:33.502470 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205f274-e26d-4f94-a642-f1d1f4ca2d97 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:02:33.503981 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-4205f274-e26d-4f94-a642-f1d1f4ca2d97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:02:33.522254 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205f274-e26d-4f94-a642-f1d1f4ca2d97 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:02:33.528128 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c2158-f845-4610-b4b9-3aa60161b7f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:02:33.528782 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c2158-f845-4610-b4b9-3aa60161b7f8 service nova] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:02:33.529064 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c2158-f845-4610-b4b9-3aa60161b7f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:02:33.529337 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c2158-f845-4610-b4b9-3aa60161b7f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:02:33.531139 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e22c2158-f845-4610-b4b9-3aa60161b7f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:02:33.545592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c2158-f845-4610-b4b9-3aa60161b7f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:02:33.547345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c2158-f845-4610-b4b9-3aa60161b7f8 service nova] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:02:33.570131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1473/2920] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 13:02:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:02:41.108883 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=9 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 13:02:43.555024 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=12 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 13:02:44.831161 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deea5a-0299-434c-9937-0c029176a4b7 service placement] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:02:44.849978 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deea5a-0299-434c-9937-0c029176a4b7 service placement] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:02:44.856079 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00538c40-9706-415c-9788-448769ef7628 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccf2dedaf7a449d2b51b9e9f3b420091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:02:44.856771 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00538c40-9706-415c-9788-448769ef7628 service nova] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:02:44.857083 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00538c40-9706-415c-9788-448769ef7628 service nova] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:02:44.857334 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00538c40-9706-415c-9788-448769ef7628 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:02:44.859222 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-00538c40-9706-415c-9788-448769ef7628 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:02:44.872754 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00538c40-9706-415c-9788-448769ef7628 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:02:44.874397 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00538c40-9706-415c-9788-448769ef7628 service nova] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:02:44.885589 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1448/2921] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 13:02:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:02:47.706113 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c2158-f845-4610-b4b9-3aa60161b7f8 service nova] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:02:47.715614 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c2158-f845-4610-b4b9-3aa60161b7f8 service nova] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:02:47.719214 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4e577-6d38-4de3-b416-55cb39b2a0f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:02:47.719849 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4e577-6d38-4de3-b416-55cb39b2a0f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:02:47.720131 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4e577-6d38-4de3-b416-55cb39b2a0f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:02:47.720408 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4e577-6d38-4de3-b416-55cb39b2a0f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:02:47.722814 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-94d4e577-6d38-4de3-b416-55cb39b2a0f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:02:47.784958 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4e577-6d38-4de3-b416-55cb39b2a0f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:02:47.786462 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4e577-6d38-4de3-b416-55cb39b2a0f5 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:02:47.796185 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1474/2922] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 13:02:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:02:54.880763 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=6 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 13:02:57.791818 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=30 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 13:03:51.478604 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00538c40-9706-415c-9788-448769ef7628 service nova] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:03:51.480044 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-00538c40-9706-415c-9788-448769ef7628 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:03:51.494682 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00538c40-9706-415c-9788-448769ef7628 service nova] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:03:51.499843 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eac9a-6812-415a-adff-6a63b9b2b2d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:03:51.500494 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eac9a-6812-415a-adff-6a63b9b2b2d7 service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:03:51.500768 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eac9a-6812-415a-adff-6a63b9b2b2d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:03:51.501043 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eac9a-6812-415a-adff-6a63b9b2b2d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:03:51.510236 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eac9a-6812-415a-adff-6a63b9b2b2d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:03:51.511884 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eac9a-6812-415a-adff-6a63b9b2b2d7 service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:03:51.522952 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1449/2923] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 13:03:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:04:01.517960 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=6 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 13:04:52.193246 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4e577-6d38-4de3-b416-55cb39b2a0f5 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:04:52.203110 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4e577-6d38-4de3-b416-55cb39b2a0f5 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:04:52.206959 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a299f7-2847-4c80-878b-22a32e335ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '375d321d6de2494794254e50151c02c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:04:52.207537 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a299f7-2847-4c80-878b-22a32e335ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:04:52.207841 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a299f7-2847-4c80-878b-22a32e335ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:04:52.208116 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a299f7-2847-4c80-878b-22a32e335ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:04:52.210018 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-94a299f7-2847-4c80-878b-22a32e335ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:04:52.273099 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a299f7-2847-4c80-878b-22a32e335ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:04:52.274592 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a299f7-2847-4c80-878b-22a32e335ab2 service neutron] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:04:52.284258 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1475/2924] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 13:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:04:55.469380 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eac9a-6812-415a-adff-6a63b9b2b2d7 service placement] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:04:55.486937 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eac9a-6812-415a-adff-6a63b9b2b2d7 service placement] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:04:55.492460 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87dffa-4c04-4b4b-8266-0703f64cce89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:04:55.493400 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87dffa-4c04-4b4b-8266-0703f64cce89 service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:04:55.493680 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87dffa-4c04-4b4b-8266-0703f64cce89 service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:04:55.493984 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87dffa-4c04-4b4b-8266-0703f64cce89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:04:55.495906 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2d87dffa-4c04-4b4b-8266-0703f64cce89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:04:55.508646 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87dffa-4c04-4b4b-8266-0703f64cce89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:04:55.510285 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87dffa-4c04-4b4b-8266-0703f64cce89 service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:04:55.529303 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1450/2925] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 13:04:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:04:58.636156 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a299f7-2847-4c80-878b-22a32e335ab2 service neutron] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:04:58.663952 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a299f7-2847-4c80-878b-22a32e335ab2 service neutron] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:04:58.671050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcd6c6-f28a-4653-a4fc-d6730240a1f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:04:58.671050 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcd6c6-f28a-4653-a4fc-d6730240a1f6 service placement] REQUEST_METHOD: `GET` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:04:58.672053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87dffa-4c04-4b4b-8266-0703f64cce89 service placement] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:04:58.672598 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcd6c6-f28a-4653-a4fc-d6730240a1f6 service placement] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:04:58.672893 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcd6c6-f28a-4653-a4fc-d6730240a1f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:04:58.673831 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-2d87dffa-4c04-4b4b-8266-0703f64cce89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:04:58.674871 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-e0dcd6c6-f28a-4653-a4fc-d6730240a1f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:04:58.688728 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcd6c6-f28a-4653-a4fc-d6730240a1f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:04:58.690933 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcd6c6-f28a-4653-a4fc-d6730240a1f6 service placement] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:04:58.692053 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87dffa-4c04-4b4b-8266-0703f64cce89 service placement] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:04:58.697005 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:04:58.698517 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:04:58.699015 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:04:58.699395 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] PATH_INFO: `/v3/users/318c288d31a74072b61ef089a028133d` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:04:58.700839 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=318c288d31a74072b61ef089a028133d)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:04:58.703119 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1476/2926] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 13:04:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:04:58.704348 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:04:58.761244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '318c288d31a74072b61ef089a028133d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:58.761899 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '318c288d31a74072b61ef089a028133d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:58.762544 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '318c288d31a74072b61ef089a028133d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:58.771365 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '318c288d31a74072b61ef089a028133d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:58.781855 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '318c288d31a74072b61ef089a028133d'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:58.806244 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1451/2927] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 13:04:58 2022] DELETE /identity/v3/users/318c288d31a74072b61ef089a028133d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 13:04:58.984345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcd6c6-f28a-4653-a4fc-d6730240a1f6 service placement] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:04:58.995332 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcd6c6-f28a-4653-a4fc-d6730240a1f6 service placement] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:04:59.053207 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:04:59.054035 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:04:59.054324 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:04:59.054604 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] PATH_INFO: `/v3/projects/c09181aee1194e42a4e88b4e37592e1c` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:04:59.055783 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c09181aee1194e42a4e88b4e37592e1c)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:04:59.057689 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:04:59.103225 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c09181aee1194e42a4e88b4e37592e1c', 'request_id': 'req-71a07cd9-f3a6-4532-bf3a-668074e5b21e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.110379 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c09181aee1194e42a4e88b4e37592e1c', 'request_id': 'req-71a07cd9-f3a6-4532-bf3a-668074e5b21e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.117056 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c09181aee1194e42a4e88b4e37592e1c', 'request_id': 'req-71a07cd9-f3a6-4532-bf3a-668074e5b21e'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.127654 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] The token cache is being invalidate because project c09181aee1194e42a4e88b4e37592e1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 13:04:59.127941 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.128564 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.130174 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1477/2928] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 13:04:58 2022] DELETE /identity/v3/projects/c09181aee1194e42a4e88b4e37592e1c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 13:04:59.134189 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:04:59.135807 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:04:59.149154 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614c6a3-a384-48fe-8905-2606079aae84 admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:04:59.206428 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:04:59.207302 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:04:59.207669 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:04:59.207845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] PATH_INFO: `/v3/users/134a324d31944b3a8fe7f9d3fb58a09c` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:04:59.209065 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] RBAC: Authorizing `identity:delete_user(user_id=134a324d31944b3a8fe7f9d3fb58a09c)` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:04:59.211009 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:04:59.273636 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '134a324d31944b3a8fe7f9d3fb58a09c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.274311 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '134a324d31944b3a8fe7f9d3fb58a09c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.274906 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '134a324d31944b3a8fe7f9d3fb58a09c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.281838 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '134a324d31944b3a8fe7f9d3fb58a09c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.286119 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 13:04:59.292150 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.notifications [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '134a324d31944b3a8fe7f9d3fb58a09c'} {{(pid=80326) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.313616 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1452/2929] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Jul 25 13:04:59 2022] DELETE /identity/v3/users/134a324d31944b3a8fe7f9d3fb58a09c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 13:04:59.490626 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] Authenticating user token {{(pid=80327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:04:59.492060 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.fernet_utils [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:04:59.505518 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a07cd9-f3a6-4532-bf3a-668074e5b21e admin admin] Validating token access rules against request {{(pid=80327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:04:59.515023 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 13:04:59.564090 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c6028-74e2-44c3-97ca-d9a3c6f89189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bca01c7266944329ecb3cbe0887bc8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61157336119141239b389248fadfbdf0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:04:59.564954 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c6028-74e2-44c3-97ca-d9a3c6f89189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:04:59.565825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c6028-74e2-44c3-97ca-d9a3c6f89189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:04:59.566124 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c6028-74e2-44c3-97ca-d9a3c6f89189 admin admin] PATH_INFO: `/v3/projects/02eb52c6faa844d6a087c371b796c04e` {{(pid=80327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:04:59.567304 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c6028-74e2-44c3-97ca-d9a3c6f89189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02eb52c6faa844d6a087c371b796c04e)` {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:04:59.569221 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c6028-74e2-44c3-97ca-d9a3c6f89189 admin admin] RBAC: Authorization granted {{(pid=80327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:04:59.615384 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-016c6028-74e2-44c3-97ca-d9a3c6f89189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02eb52c6faa844d6a087c371b796c04e', 'request_id': 'req-016c6028-74e2-44c3-97ca-d9a3c6f89189'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.622191 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-016c6028-74e2-44c3-97ca-d9a3c6f89189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02eb52c6faa844d6a087c371b796c04e', 'request_id': 'req-016c6028-74e2-44c3-97ca-d9a3c6f89189'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.628825 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-016c6028-74e2-44c3-97ca-d9a3c6f89189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02eb52c6faa844d6a087c371b796c04e', 'request_id': 'req-016c6028-74e2-44c3-97ca-d9a3c6f89189'} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.638931 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-016c6028-74e2-44c3-97ca-d9a3c6f89189 admin admin] The token cache is being invalidate because project 02eb52c6faa844d6a087c371b796c04e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 13:04:59.639499 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-016c6028-74e2-44c3-97ca-d9a3c6f89189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.639824 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG keystone.notifications [None req-016c6028-74e2-44c3-97ca-d9a3c6f89189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 13:04:59.641345 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: [pid: 80327|app: 0|req: 1478/2930] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Jul 25 13:04:59 2022] DELETE /identity/v3/projects/02eb52c6faa844d6a087c371b796c04e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 13:05:09.309663 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 13:05:09.636087 dv-cinder-825492-28-iscsi devstack@keystone.service[80327]: DEBUG dbcounter [-] [80327] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 13:08:53.388510 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] Authenticating user token {{(pid=80326) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 13:08:53.389944 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:08:53.551483 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a6fc2-1657-4e56-b0f8-c452b0f7544c admin admin] Validating token access rules against request {{(pid=80326) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 13:08:53.609845 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769a672-1119-4094-8317-0d92c70cd55f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81037f61e01e4a93982503720f7404e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ff5a41f8ca6438295c512792d2022e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80326) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 13:08:53.610533 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769a672-1119-4094-8317-0d92c70cd55f service placement] REQUEST_METHOD: `GET` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 13:08:53.610811 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769a672-1119-4094-8317-0d92c70cd55f service placement] SCRIPT_NAME: `/identity` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 13:08:53.611086 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769a672-1119-4094-8317-0d92c70cd55f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80326) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 13:08:53.612891 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.fernet_utils [None req-5769a672-1119-4094-8317-0d92c70cd55f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80326) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 13:08:53.624509 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769a672-1119-4094-8317-0d92c70cd55f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 13:08:53.626114 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769a672-1119-4094-8317-0d92c70cd55f service placement] RBAC: Authorization granted {{(pid=80326) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 13:08:53.636527 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: [pid: 80326|app: 0|req: 1453/2931] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Jul 25 13:08:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 13:09:03.631714 dv-cinder-825492-28-iscsi devstack@keystone.service[80326]: DEBUG dbcounter [-] [80326] Writing DB stats keystone:SELECT=89 {{(pid=80326) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}